Jul 16 21:44:42.697812 ip-10-0-136-31 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 21:44:42.697825 ip-10-0-136-31 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 21:44:42.697835 ip-10-0-136-31 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 21:44:42.698266 ip-10-0-136-31 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 21:44:52.867264 ip-10-0-136-31 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 21:44:52.867283 ip-10-0-136-31 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c9ca669bed694e07a6577c9a00572bd5 -- Jul 16 21:46:44.563416 ip-10-0-136-31 systemd[1]: Starting Kubernetes Kubelet... Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.940321 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945643 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945659 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945664 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945669 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945675 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945679 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945683 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:46:45.018775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945687 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:46:45.008011 ip-10-0-136-31 systemd[1]: Started Kubernetes Kubelet. Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945691 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945695 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945701 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945707 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945710 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945714 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945718 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945727 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945732 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945736 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945741 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945745 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945749 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945753 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945757 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945761 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945764 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945768 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:46:45.020530 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945772 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945776 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945779 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945783 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945787 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945790 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945794 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945799 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945803 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945807 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945811 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945815 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945819 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945823 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945826 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945830 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945833 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945837 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945840 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945844 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:46:45.021619 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945848 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945853 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945858 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945863 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945867 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945871 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945875 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945879 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945884 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945888 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945892 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945896 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945900 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945903 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945908 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945911 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945915 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945919 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945923 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:46:45.022686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945927 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945931 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945934 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945938 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945944 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945949 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945953 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945957 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945961 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945965 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945969 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945974 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945978 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945981 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.945985 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.946039 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948039 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948230 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948249 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948253 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:46:45.023549 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948256 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948260 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948262 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948271 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948275 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948278 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948281 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948284 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948287 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948291 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948294 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948297 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948300 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.948303 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949058 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949066 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949069 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949073 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949076 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949079 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:46:45.024604 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949088 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949092 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949095 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949097 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949101 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949104 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949107 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949109 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949117 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949120 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949123 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949126 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949129 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949131 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949136 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949158 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949164 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949168 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949173 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949177 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:46:45.025686 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949180 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949183 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949185 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949188 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949191 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949194 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949197 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949200 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949203 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949206 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949209 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949212 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949215 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949217 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949220 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949222 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949225 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949227 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949231 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:46:45.026707 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949234 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949236 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949239 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949241 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949244 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949247 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949251 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949254 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949257 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949259 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949262 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949264 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949267 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949269 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949272 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949274 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949277 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949279 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949282 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949284 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:46:45.027597 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949287 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949290 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949292 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949295 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949298 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949301 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949303 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949305 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949309 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949312 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949316 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949318 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949321 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949323 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949326 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949329 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949331 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949334 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949336 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949339 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:46:45.028651 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949341 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949344 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949348 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949351 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949354 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949356 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949359 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949361 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949364 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949366 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949369 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949371 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949374 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949378 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949747 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949752 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949754 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949757 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949760 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:46:45.029860 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949763 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949766 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949768 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949771 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949773 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949776 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949778 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949781 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949784 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949786 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949789 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949793 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949797 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949800 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949802 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949805 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949807 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949810 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949812 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:46:45.030719 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949815 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949818 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949821 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949823 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949826 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949829 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949832 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949836 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949838 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949841 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949846 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949849 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949851 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949854 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949859 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949862 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949865 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949868 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:46:45.031596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949871 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949875 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949879 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949882 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949885 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949888 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949891 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949894 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949896 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949899 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949901 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949904 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949907 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949910 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949912 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949915 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949917 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949920 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949922 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:46:45.033624 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949925 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949927 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949931 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949935 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949937 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949941 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949943 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949946 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949948 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949951 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949954 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949956 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949959 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949961 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949964 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949966 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949969 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949971 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949974 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949976 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:46:45.035317 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949979 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949981 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949984 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949986 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949989 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949992 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949994 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949997 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.949999 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950002 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950004 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950007 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950010 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950012 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950015 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950019 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950021 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950025 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950102 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950109 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950114 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 21:46:45.037621 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950118 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950122 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950125 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950129 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950133 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950137 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950152 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950156 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950159 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950162 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950166 2563 flags.go:64] FLAG: --cgroup-root="" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950169 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950172 2563 flags.go:64] FLAG: --client-ca-file="" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950175 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950177 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950182 2563 flags.go:64] FLAG: --cluster-domain="" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950184 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950187 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950191 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950194 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950202 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950205 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950208 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 21:46:45.039860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950212 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950215 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950218 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950222 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950226 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950230 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950234 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950237 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950240 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950243 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950246 2563 flags.go:64] FLAG: --enable-server="true" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950249 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950253 2563 flags.go:64] FLAG: --event-burst="100" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950256 2563 flags.go:64] FLAG: --event-qps="50" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950259 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950262 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950265 2563 flags.go:64] FLAG: --eviction-hard="" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950268 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950271 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950274 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950277 2563 flags.go:64] FLAG: --eviction-soft="" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950280 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950283 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950286 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950289 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 21:46:45.042341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950292 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950295 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950298 2563 flags.go:64] FLAG: --feature-gates="" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950301 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950304 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950307 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950310 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950313 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950316 2563 flags.go:64] FLAG: --help="false" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950319 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950323 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950326 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950331 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950335 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950338 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950341 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950344 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950347 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950349 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950352 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950355 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950358 2563 flags.go:64] FLAG: --kube-reserved="" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950361 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950364 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 21:46:45.044949 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950367 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950370 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950373 2563 flags.go:64] FLAG: --lock-file="" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950375 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950378 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950381 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950386 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950389 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950392 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950395 2563 flags.go:64] FLAG: --logging-format="text" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950398 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950401 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950404 2563 flags.go:64] FLAG: --manifest-url="" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950407 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950411 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950414 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950418 2563 flags.go:64] FLAG: --max-pods="110" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950420 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950425 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950428 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950432 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950435 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950438 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950441 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950447 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 21:46:45.046749 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950450 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950453 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950457 2563 flags.go:64] FLAG: --pod-cidr="" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950459 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950465 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950468 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950471 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950473 2563 flags.go:64] FLAG: --port="10250" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950477 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950480 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dde6ad15c33651de" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950483 2563 flags.go:64] FLAG: --qos-reserved="" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950485 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950488 2563 flags.go:64] FLAG: --register-node="true" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950491 2563 flags.go:64] FLAG: --register-with-taints="" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950495 2563 flags.go:64] FLAG: --registry-burst="10" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950498 2563 flags.go:64] FLAG: --registry-qps="5" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950501 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950503 2563 flags.go:64] FLAG: --reserved-memory="" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950507 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950510 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950513 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950516 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950518 2563 flags.go:64] FLAG: --runonce="false" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950521 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950524 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 21:46:45.050672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950529 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950532 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950534 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950539 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950542 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950546 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950549 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950552 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950554 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950558 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950561 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950564 2563 flags.go:64] FLAG: --system-cgroups="" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950566 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950571 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950574 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950577 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950581 2563 flags.go:64] FLAG: --tls-min-version="" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950584 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950587 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950590 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950593 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950596 2563 flags.go:64] FLAG: --v="2" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950600 2563 flags.go:64] FLAG: --version="false" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950604 2563 flags.go:64] FLAG: --vmodule="" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950607 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 21:46:45.052170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.950611 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950721 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950725 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950728 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950731 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950734 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950737 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950740 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950743 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950746 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950750 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950753 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950756 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950759 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950761 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950764 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950766 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950769 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950771 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950774 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:46:45.059199 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950776 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950779 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950781 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950784 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950786 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950789 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950791 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950794 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950796 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950800 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950802 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950805 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950808 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950811 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950813 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950815 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950818 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950820 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950823 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950827 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:46:45.059914 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950830 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950832 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950837 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950839 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950842 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950845 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950847 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950850 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950853 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950855 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950858 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950861 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950863 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950866 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950868 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950871 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950873 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950877 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950881 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:46:45.060663 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950884 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950886 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950889 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950891 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950894 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950896 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950899 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950902 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950906 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950909 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950912 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950915 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950920 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950924 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950927 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950931 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950934 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950937 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950940 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:46:45.061617 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950943 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950946 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950948 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950951 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950953 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950956 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950958 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950961 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950963 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950966 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950968 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950971 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950974 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950976 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950979 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950981 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950983 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950986 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950989 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950991 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:46:45.062467 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950994 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.950996 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.951002 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.957578 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.957593 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.958398 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.958409 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958469 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958474 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958477 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958480 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958483 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958492 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958496 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958499 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:46:45.063552 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958501 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958504 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958507 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958510 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958512 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958515 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958518 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958520 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958523 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958525 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958528 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958531 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958533 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958536 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958538 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958541 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958543 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958546 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958548 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958551 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:46:45.064408 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958554 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958558 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958563 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958565 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958568 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958571 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958573 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958576 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958579 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958581 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958590 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958593 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958596 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958598 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958601 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958604 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958606 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958608 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958611 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958613 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:46:45.065443 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958616 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958619 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958621 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958624 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958627 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958629 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958632 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958634 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958637 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958639 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958642 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958644 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958647 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958651 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958655 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958658 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958660 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958663 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958666 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958668 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:46:45.066493 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958671 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958675 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958679 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958686 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958689 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958693 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958697 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958700 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958703 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958706 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958709 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958712 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958715 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958717 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958720 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958723 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958726 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958729 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958732 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:46:45.067713 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958734 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958737 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958740 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958743 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958745 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958749 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958753 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958758 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958760 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958763 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958766 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958768 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.958773 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958911 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958916 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 16 21:46:45.068753 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958919 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958921 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958932 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958935 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958938 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958940 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958942 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958945 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958948 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958951 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958953 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958956 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958958 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958963 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958966 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958969 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958972 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958974 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958977 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958980 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 21:46:45.069596 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958982 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958985 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958987 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958992 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958994 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.958999 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959001 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959004 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959006 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959009 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959012 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959014 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959017 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959020 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959024 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959032 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959035 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959038 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959040 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 21:46:45.070728 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959043 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959045 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959048 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959050 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959053 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959055 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959058 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959060 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959063 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959065 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959067 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959070 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959072 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959075 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959077 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959080 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959084 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959087 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959091 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959094 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 21:46:45.071595 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959096 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959099 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959102 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959104 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959106 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959110 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959112 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959115 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959117 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959125 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959128 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959130 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959133 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959135 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959138 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959154 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959157 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959159 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959162 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959164 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 21:46:45.072326 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959167 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959169 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959172 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959174 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959177 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959179 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959182 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959185 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959189 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959191 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959195 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959198 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959201 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959204 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959207 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959210 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959213 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 21:46:45.072917 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:44.959216 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.959221 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.959327 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.961356 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.962587 2563 server.go:1019] "Starting client certificate rotation" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.962704 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.962741 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.965750 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.968958 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.978256 2563 log.go:25] "Validated CRI v1 runtime API" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.980331 2563 log.go:25] "Validated CRI v1 image API" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.981159 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 21:46:45.073800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.983809 2563 fs.go:135] Filesystem UUIDs: map[11f9e0ad-b819-4462-80da-3cf465ba999c:/dev/nvme0n1p3 5e50f602-64d2-4200-869e-976cfa4b3b01:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 16 21:46:45.074438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.983823 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 21:46:45.074643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.989488 2563 manager.go:216] Machine: {Timestamp:2026-07-16 21:46:44.988194115 +0000 UTC m=+0.321273157 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100109 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec204eca51bdcd10c3a28f91902abfcd SystemUUID:ec204eca-51bd-cd10-c3a2-8f91902abfcd BootID:c9ca669b-ed69-4e07-a657-7c9a00572bd5 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7f:4e:64:28:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7f:4e:64:28:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:60:6b:8f:65:e2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 21:46:45.074643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.989581 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 21:46:45.074643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.989649 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 21:46:45.074643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.992540 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 21:46:45.074643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.992565 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-31.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 21:46:45.074643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.993717 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 21:46:45.074643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.993727 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 21:46:45.074643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.993735 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.994318 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.995584 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.998604 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.998718 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.998909 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.998920 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.998930 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.998941 2563 kubelet.go:398] "Adding apiserver pod source" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.998949 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.999326 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:44.999342 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.001373 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.002722 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.002750 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003341 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003355 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003360 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003366 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003375 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003384 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003413 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003419 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003427 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003433 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003442 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.003449 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.004160 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.004172 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.005361 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.007224 2563 server.go:1268] "Started kubelet" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.007296 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.007320 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.007392 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.009622 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.011696 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.014124 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-31.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.014158 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.014159 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-31.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 21:46:45.155541 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.014182 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.017642 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-31.ec2.internal.18c2e360451036dc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-31.ec2.internal,UID:ip-10-0-136-31.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-31.ec2.internal,},FirstTimestamp:2026-07-16 21:46:45.00719382 +0000 UTC m=+0.340272841,LastTimestamp:2026-07-16 21:46:45.00719382 +0000 UTC m=+0.340272841,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-31.ec2.internal,}" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.018538 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.018552 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.018941 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.019534 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.019559 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.019689 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.019697 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020003 2563 factory.go:55] Registering systemd factory Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020018 2563 factory.go:223] Registration of the systemd container factory successfully Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020070 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020269 2563 factory.go:153] Registering CRI-O factory Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020285 2563 factory.go:223] Registration of the crio container factory successfully Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020328 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020350 2563 factory.go:103] Registering Raw factory Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020364 2563 manager.go:1196] Started watching for new ooms in manager Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.020925 2563 manager.go:349] Starting recovery of all containers Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.031330 2563 manager.go:354] Recovery completed Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.033218 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-31.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.033278 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.036598 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.045384 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.045435 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.045449 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.046013 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.046025 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.046042 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.046025 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.047055 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.047080 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.047093 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.047214 2563 policy_none.go:49] "None policy: Start" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.047231 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 21:46:45.172020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.047244 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.048123 2563 policy_none.go:47] "Start" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.059211 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-31.ec2.internal.18c2e3604757914e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-31.ec2.internal,UID:ip-10-0-136-31.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-31.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-31.ec2.internal,},FirstTimestamp:2026-07-16 21:46:45.045424462 +0000 UTC m=+0.378503484,LastTimestamp:2026-07-16 21:46:45.045424462 +0000 UTC m=+0.378503484,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-31.ec2.internal,}" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.078668 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-31.ec2.internal.18c2e3604757d399 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-31.ec2.internal,UID:ip-10-0-136-31.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-31.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-31.ec2.internal,},FirstTimestamp:2026-07-16 21:46:45.045441433 +0000 UTC m=+0.378520456,LastTimestamp:2026-07-16 21:46:45.045441433 +0000 UTC m=+0.378520456,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-31.ec2.internal,}" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.084760 2563 manager.go:337] "Starting Device Plugin manager" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.084784 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.084793 2563 server.go:87] "Starting device plugin registration server" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.085022 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.085034 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.085111 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.085202 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.085212 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.085790 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.085830 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.093781 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-31.ec2.internal.18c2e360475803dd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-31.ec2.internal,UID:ip-10-0-136-31.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-31.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-31.ec2.internal,},FirstTimestamp:2026-07-16 21:46:45.045453789 +0000 UTC m=+0.378532811,LastTimestamp:2026-07-16 21:46:45.045453789 +0000 UTC m=+0.378532811,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-31.ec2.internal,}" Jul 16 21:46:45.173060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.098873 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pkcxd" Jul 16 21:46:45.174095 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.106457 2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-31.ec2.internal.18c2e3604757914e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-31.ec2.internal.18c2e3604757914e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-31.ec2.internal,UID:ip-10-0-136-31.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-31.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-31.ec2.internal,},FirstTimestamp:2026-07-16 21:46:45.045424462 +0000 UTC m=+0.378503484,LastTimestamp:2026-07-16 21:46:45.047066363 +0000 UTC m=+0.380145385,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-31.ec2.internal,}" Jul 16 21:46:45.174095 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.112702 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pkcxd" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.185317 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.186348 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.186378 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.186392 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.186421 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.196181 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.196199 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-31.ec2.internal\": node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.232633 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.233849 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.233893 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.233924 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.233973 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.237876 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:46:45.257331 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.240459 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.334379 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal"] Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.334444 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.335916 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.335942 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.335952 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337116 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337244 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337268 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337809 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337821 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337830 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337840 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337842 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.337852 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.338837 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.338857 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.339466 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.339488 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.339499 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.340760 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.365679 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-31.ec2.internal\" not found" node="ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.375775 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.370065 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-31.ec2.internal\" not found" node="ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.463448 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.421796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a79463808a4d0705f9376bbce08cef7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal\" (UID: \"0a79463808a4d0705f9376bbce08cef7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.463448 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.421823 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a79463808a4d0705f9376bbce08cef7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal\" (UID: \"0a79463808a4d0705f9376bbce08cef7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.463448 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.421840 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c1faed2aff0ab49640344e2b4b87931f-config\") pod \"kube-apiserver-proxy-ip-10-0-136-31.ec2.internal\" (UID: \"c1faed2aff0ab49640344e2b4b87931f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.463448 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.445469 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.577572 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.522584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a79463808a4d0705f9376bbce08cef7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal\" (UID: \"0a79463808a4d0705f9376bbce08cef7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.577572 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.522609 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a79463808a4d0705f9376bbce08cef7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal\" (UID: \"0a79463808a4d0705f9376bbce08cef7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.577572 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.522625 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c1faed2aff0ab49640344e2b4b87931f-config\") pod \"kube-apiserver-proxy-ip-10-0-136-31.ec2.internal\" (UID: \"c1faed2aff0ab49640344e2b4b87931f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.577572 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.522660 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c1faed2aff0ab49640344e2b4b87931f-config\") pod \"kube-apiserver-proxy-ip-10-0-136-31.ec2.internal\" (UID: \"c1faed2aff0ab49640344e2b4b87931f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.577572 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.522696 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0a79463808a4d0705f9376bbce08cef7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal\" (UID: \"0a79463808a4d0705f9376bbce08cef7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.577572 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.522720 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0a79463808a4d0705f9376bbce08cef7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal\" (UID: \"0a79463808a4d0705f9376bbce08cef7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.577572 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.546401 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.695860 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.647338 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.695860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.667770 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.695860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.674056 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" Jul 16 21:46:45.823703 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.748417 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:45.923826 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.849556 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:46.031528 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:45.949923 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:46.031528 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.961931 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 21:46:46.031528 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:45.962077 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 21:46:46.121123 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:46.051005 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:46.121123 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.070424 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:46:46.121123 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.112334 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 21:46:46.121123 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.113829 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientMemory" Jul 16 21:46:46.121123 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.113866 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 21:46:46.121123 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.113880 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeHasSufficientPID" Jul 16 21:46:46.121123 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.117257 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 21:41:45 +0000 UTC" deadline="2028-04-04 03:17:13.908247664 +0000 UTC" Jul 16 21:46:46.121123 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.117279 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15053h30m27.790971403s" Jul 16 21:46:46.211525 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:46.151125 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-31.ec2.internal\" not found" Jul 16 21:46:46.211525 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.177205 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:46:46.211525 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:46.193418 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1faed2aff0ab49640344e2b4b87931f.slice/crio-9ccadb98e66c39a68930c69c7a55b36f8edfa08decca74bf52ee04f479e16497 WatchSource:0}: Error finding container 9ccadb98e66c39a68930c69c7a55b36f8edfa08decca74bf52ee04f479e16497: Status 404 returned error can't find the container with id 9ccadb98e66c39a68930c69c7a55b36f8edfa08decca74bf52ee04f479e16497 Jul 16 21:46:46.211525 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.199751 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 21:46:46.219687 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.219670 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" Jul 16 21:46:46.226031 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:46.226004 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a79463808a4d0705f9376bbce08cef7.slice/crio-9825ea583394073acddac12dd58c2a707451ec550adcf42f996e1145991348c2 WatchSource:0}: Error finding container 9825ea583394073acddac12dd58c2a707451ec550adcf42f996e1145991348c2: Status 404 returned error can't find the container with id 9825ea583394073acddac12dd58c2a707451ec550adcf42f996e1145991348c2 Jul 16 21:46:46.235645 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.235626 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 21:46:46.236715 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.236698 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" Jul 16 21:46:46.237440 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.237403 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" event={"ID":"0a79463808a4d0705f9376bbce08cef7","Type":"ContainerStarted","Data":"9825ea583394073acddac12dd58c2a707451ec550adcf42f996e1145991348c2"} Jul 16 21:46:46.238380 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.238359 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" event={"ID":"c1faed2aff0ab49640344e2b4b87931f","Type":"ContainerStarted","Data":"9ccadb98e66c39a68930c69c7a55b36f8edfa08decca74bf52ee04f479e16497"} Jul 16 21:46:46.248632 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.248611 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 21:46:46.363976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.363891 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:46:46.938858 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:46.938713 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 21:46:47.000422 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.000250 2563 apiserver.go:52] "Watching apiserver" Jul 16 21:46:47.010546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.010519 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 21:46:47.011046 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.011020 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-mjhl2","openshift-network-diagnostics/network-check-target-8qd4b","openshift-ovn-kubernetes/ovnkube-node-7xmnr","kube-system/konnectivity-agent-6s8l9","openshift-dns/node-resolver-fpm78","openshift-image-registry/node-ca-pzmdz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal","openshift-multus/network-metrics-daemon-xm229","openshift-network-operator/iptables-alerter-sv64j","kube-system/global-pull-secret-syncer-5p9fm","kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc","openshift-cluster-node-tuning-operator/tuned-g6cwc","openshift-multus/multus-additional-cni-plugins-btfm6"] Jul 16 21:46:47.013589 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.013551 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.017171 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.016484 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.019160 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.019138 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 21:46:47.019282 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.019215 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.019352 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.019333 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 21:46:47.020176 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020158 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 21:46:47.020815 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020425 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n866j\"" Jul 16 21:46:47.020815 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020471 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 21:46:47.020815 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020522 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 21:46:47.020815 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020631 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 21:46:47.020815 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020744 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 21:46:47.021122 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020842 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 21:46:47.021122 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020896 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 21:46:47.021122 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.020938 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l4phh\"" Jul 16 21:46:47.021454 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.021433 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:47.021553 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.021523 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:46:47.022206 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.022183 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pbw5s\"" Jul 16 21:46:47.022616 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.022599 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 21:46:47.022926 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.022907 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:46:47.023240 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.023219 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 21:46:47.023806 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.023786 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.025954 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.025934 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.026580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.026561 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 21:46:47.026666 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.026639 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8rvzk\"" Jul 16 21:46:47.028275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.028255 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:46:47.029166 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.029144 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 21:46:47.029597 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.029577 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r9k2x\"" Jul 16 21:46:47.029693 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.029662 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 21:46:47.031009 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.030985 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.032423 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032391 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-host\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032439 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fa707431-f054-46aa-a35e-2f743c19bdda-iptables-alerter-script\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032468 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-serviceca\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032518 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-system-cni-dir\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032549 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032589 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lbdh\" (UniqueName: \"kubernetes.io/projected/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-kube-api-access-2lbdh\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-cnibin\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032645 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-os-release\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032675 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69ct9\" (UniqueName: \"kubernetes.io/projected/54b92567-7296-4fce-b493-dd41959095ce-kube-api-access-69ct9\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032700 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fa707431-f054-46aa-a35e-2f743c19bdda-host-slash\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032746 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6nj7\" (UniqueName: \"kubernetes.io/projected/ef310c53-4c7a-4331-91b9-6e7a2bb27841-kube-api-access-f6nj7\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032779 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032804 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.033019 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.033708 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032875 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.033708 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.032919 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9xl8\" (UniqueName: \"kubernetes.io/projected/fa707431-f054-46aa-a35e-2f743c19bdda-kube-api-access-b9xl8\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.033708 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.033591 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.033708 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.033674 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:46:47.037312 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.036740 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 21:46:47.037312 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.037024 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 21:46:47.037312 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.037051 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.037312 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.037096 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 21:46:47.038273 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.038251 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 21:46:47.038598 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.038530 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 21:46:47.038685 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.038626 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 21:46:47.038748 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.038727 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 21:46:47.038889 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.038860 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kxj2v\"" Jul 16 21:46:47.039045 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.039027 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 21:46:47.039180 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.039158 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cvdww\"" Jul 16 21:46:47.040138 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.040086 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 21:46:47.041153 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.040117 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 21:46:47.041260 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.040364 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 21:46:47.041316 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.040496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.041842 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.041798 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-knd79\"" Jul 16 21:46:47.042039 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.041998 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 21:46:47.043145 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.043072 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:47.043145 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.043118 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:46:47.044347 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.044326 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:46:47.044806 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.044716 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cbvld\"" Jul 16 21:46:47.045931 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.045911 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 21:46:47.076178 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.076152 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kh4dv" Jul 16 21:46:47.088656 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.088625 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kh4dv" Jul 16 21:46:47.121934 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.121913 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133063 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-socket-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133102 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-os-release\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133131 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-multus-certs\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fa707431-f054-46aa-a35e-2f743c19bdda-iptables-alerter-script\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133190 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-cni-bin\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133237 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-sys-fs\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10357a95-81f6-410a-94db-81289563dcfa-cni-binary-copy\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133289 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-socket-dir-parent\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133314 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-hostroot\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133338 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10357a95-81f6-410a-94db-81289563dcfa-multus-daemon-config\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133363 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-log-socket\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.135438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovn-node-metrics-cert\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133485 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81422712-0ebb-4177-a94f-11b2b36bc6ab-kubelet-config\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133511 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81422712-0ebb-4177-a94f-11b2b36bc6ab-dbus\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133543 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133570 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/132ee170-dfa0-45fa-a378-11f8a5cc9a76-tmp\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133597 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-cnibin\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133633 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-os-release\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fa707431-f054-46aa-a35e-2f743c19bdda-host-slash\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133687 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysctl-conf\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133713 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133741 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-kubelet\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133792 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-env-overrides\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133829 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-kubelet\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.136266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133861 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-ovn\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133893 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-system-cni-dir\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133933 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133960 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-etc-selinux\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.133989 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-cni-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134015 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51fea26c-fe2e-4d70-94c2-03c76aee179c-agent-certs\") pod \"konnectivity-agent-6s8l9\" (UID: \"51fea26c-fe2e-4d70-94c2-03c76aee179c\") " pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134041 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-var-lib-kubelet\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134066 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3dbc476-9c43-48c3-a2bf-78a67110c028-tmp-dir\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134095 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69ct9\" (UniqueName: \"kubernetes.io/projected/54b92567-7296-4fce-b493-dd41959095ce-kube-api-access-69ct9\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134124 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134150 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-conf-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134177 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-modprobe-d\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-kubernetes\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134245 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9xl8\" (UniqueName: \"kubernetes.io/projected/fa707431-f054-46aa-a35e-2f743c19bdda-kube-api-access-b9xl8\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.136950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134260 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fa707431-f054-46aa-a35e-2f743c19bdda-iptables-alerter-script\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134271 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-var-lib-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134306 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-registration-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134341 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-tuned\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134388 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-host\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134448 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-host\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134503 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-cnibin\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134505 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-cni-netd\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134581 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-os-release\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134631 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fa707431-f054-46aa-a35e-2f743c19bdda-host-slash\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134716 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134773 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/54b92567-7296-4fce-b493-dd41959095ce-system-cni-dir\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.134840 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.134897 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:46:47.634880421 +0000 UTC m=+2.967959430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:47.137845 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.134968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovnkube-script-lib\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135016 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-netns\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135045 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-run-netns\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135072 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-systemd-units\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135152 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lbdh\" (UniqueName: \"kubernetes.io/projected/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-kube-api-access-2lbdh\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135199 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6nj7\" (UniqueName: \"kubernetes.io/projected/ef310c53-4c7a-4331-91b9-6e7a2bb27841-kube-api-access-f6nj7\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135392 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-etc-kubernetes\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135422 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-systemd\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135477 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135503 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovnkube-config\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135537 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqflk\" (UniqueName: \"kubernetes.io/projected/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-kube-api-access-gqflk\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.138578 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-cni-multus\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51fea26c-fe2e-4d70-94c2-03c76aee179c-konnectivity-ca\") pod \"konnectivity-agent-6s8l9\" (UID: \"51fea26c-fe2e-4d70-94c2-03c76aee179c\") " pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135623 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-run\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135639 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpflq\" (UniqueName: \"kubernetes.io/projected/132ee170-dfa0-45fa-a378-11f8a5cc9a76-kube-api-access-zpflq\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135694 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-systemd\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135717 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135734 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtrf8\" (UniqueName: \"kubernetes.io/projected/e47fb831-76fb-4d1c-a7d1-bf21f1935483-kube-api-access-rtrf8\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135751 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-k8s-cni-cncf-io\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135773 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbrl2\" (UniqueName: \"kubernetes.io/projected/10357a95-81f6-410a-94db-81289563dcfa-kube-api-access-hbrl2\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135802 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysconfig\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135839 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysctl-d\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135868 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-host\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135888 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-serviceca\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135913 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nml72\" (UniqueName: \"kubernetes.io/projected/c3dbc476-9c43-48c3-a2bf-78a67110c028-kube-api-access-nml72\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.139303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135933 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-slash\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135956 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-node-log\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135972 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-device-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135976 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-cni-binary-copy\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.135988 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-system-cni-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.136005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-sys\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.136022 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-lib-modules\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.136086 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-cnibin\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.136103 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-cni-bin\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.136123 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3dbc476-9c43-48c3-a2bf-78a67110c028-hosts-file\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.136148 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-etc-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.136452 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/54b92567-7296-4fce-b493-dd41959095ce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.139997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.136648 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-serviceca\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.145340 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.145304 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 21:46:47.151343 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.151176 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69ct9\" (UniqueName: \"kubernetes.io/projected/54b92567-7296-4fce-b493-dd41959095ce-kube-api-access-69ct9\") pod \"multus-additional-cni-plugins-btfm6\" (UID: \"54b92567-7296-4fce-b493-dd41959095ce\") " pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.151992 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.151540 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lbdh\" (UniqueName: \"kubernetes.io/projected/a160cbfd-0e76-4746-9d1a-e442b9dbdff1-kube-api-access-2lbdh\") pod \"node-ca-pzmdz\" (UID: \"a160cbfd-0e76-4746-9d1a-e442b9dbdff1\") " pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.151992 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.151948 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9xl8\" (UniqueName: \"kubernetes.io/projected/fa707431-f054-46aa-a35e-2f743c19bdda-kube-api-access-b9xl8\") pod \"iptables-alerter-sv64j\" (UID: \"fa707431-f054-46aa-a35e-2f743c19bdda\") " pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.153543 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.153494 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6nj7\" (UniqueName: \"kubernetes.io/projected/ef310c53-4c7a-4331-91b9-6e7a2bb27841-kube-api-access-f6nj7\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:47.237109 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237019 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.237109 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237069 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-etc-selinux\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.237109 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237099 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-cni-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51fea26c-fe2e-4d70-94c2-03c76aee179c-agent-certs\") pod \"konnectivity-agent-6s8l9\" (UID: \"51fea26c-fe2e-4d70-94c2-03c76aee179c\") " pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-var-lib-kubelet\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237178 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237201 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3dbc476-9c43-48c3-a2bf-78a67110c028-tmp-dir\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237242 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-conf-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237298 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-modprobe-d\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-kubernetes\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.237408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237397 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-kubernetes\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.237917 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237459 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-var-lib-kubelet\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.237917 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237652 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-conf-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.237917 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237725 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.237917 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237746 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3dbc476-9c43-48c3-a2bf-78a67110c028-tmp-dir\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.237917 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.237891 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-modprobe-d\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.238168 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238060 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-etc-selinux\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.238168 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238103 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-var-lib-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.238168 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238121 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-cni-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.238168 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238132 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-registration-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.238168 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238164 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-tuned\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238169 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-var-lib-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238204 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-cni-netd\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238205 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-registration-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238252 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovnkube-script-lib\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238279 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-netns\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238305 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-run-netns\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238332 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238341 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-netns\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238356 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-systemd-units\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238394 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-systemd-units\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238394 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-etc-kubernetes\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238433 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-systemd\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.238467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238465 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovnkube-config\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238494 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqflk\" (UniqueName: \"kubernetes.io/projected/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-kube-api-access-gqflk\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238519 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-cni-multus\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238526 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238543 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51fea26c-fe2e-4d70-94c2-03c76aee179c-konnectivity-ca\") pod \"konnectivity-agent-6s8l9\" (UID: \"51fea26c-fe2e-4d70-94c2-03c76aee179c\") " pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238279 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-cni-netd\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238567 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-run\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238588 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-run-netns\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238635 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-systemd\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238434 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-etc-kubernetes\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238592 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpflq\" (UniqueName: \"kubernetes.io/projected/132ee170-dfa0-45fa-a378-11f8a5cc9a76-kube-api-access-zpflq\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238741 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-systemd\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238771 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238800 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtrf8\" (UniqueName: \"kubernetes.io/projected/e47fb831-76fb-4d1c-a7d1-bf21f1935483-kube-api-access-rtrf8\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.239124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-k8s-cni-cncf-io\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238848 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovnkube-script-lib\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238855 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbrl2\" (UniqueName: \"kubernetes.io/projected/10357a95-81f6-410a-94db-81289563dcfa-kube-api-access-hbrl2\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238858 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-systemd\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238880 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysconfig\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238905 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysctl-d\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238930 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-host\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238964 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nml72\" (UniqueName: \"kubernetes.io/projected/c3dbc476-9c43-48c3-a2bf-78a67110c028-kube-api-access-nml72\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.238990 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-slash\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-node-log\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239041 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-device-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239068 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-system-cni-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239077 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-cni-multus\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239092 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-sys\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239127 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-lib-modules\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.239846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239144 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-host\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239153 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-cnibin\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovnkube-config\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-cni-bin\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239235 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-device-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239233 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-cnibin\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239250 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3dbc476-9c43-48c3-a2bf-78a67110c028-hosts-file\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239289 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239294 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-etc-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239327 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-etc-openvswitch\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-socket-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239348 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c3dbc476-9c43-48c3-a2bf-78a67110c028-hosts-file\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239365 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-os-release\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239393 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-multus-certs\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239403 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysconfig\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.240606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239421 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-cni-bin\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239423 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-socket-dir\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239041 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-run\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239447 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/51fea26c-fe2e-4d70-94c2-03c76aee179c-konnectivity-ca\") pod \"konnectivity-agent-6s8l9\" (UID: \"51fea26c-fe2e-4d70-94c2-03c76aee179c\") " pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239511 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-sys\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239514 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysctl-d\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239548 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-cni-bin\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239553 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-lib-modules\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239565 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-system-cni-dir\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239581 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-sys-fs\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239601 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-node-log\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239607 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10357a95-81f6-410a-94db-81289563dcfa-cni-binary-copy\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239611 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-k8s-cni-cncf-io\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239609 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-slash\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239635 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-socket-dir-parent\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.241368 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239665 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-cni-bin\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239667 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-hostroot\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239675 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-multus-socket-dir-parent\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239682 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-os-release\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239697 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-hostroot\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10357a95-81f6-410a-94db-81289563dcfa-multus-daemon-config\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239729 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239761 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-run-multus-certs\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-log-socket\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239725 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e47fb831-76fb-4d1c-a7d1-bf21f1935483-sys-fs\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239798 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovn-node-metrics-cert\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81422712-0ebb-4177-a94f-11b2b36bc6ab-kubelet-config\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239833 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-log-socket\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239848 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81422712-0ebb-4177-a94f-11b2b36bc6ab-dbus\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239874 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.242128 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239900 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/132ee170-dfa0-45fa-a378-11f8a5cc9a76-tmp\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239928 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysctl-conf\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239954 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-kubelet\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.239978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-env-overrides\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240005 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-kubelet\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-ovn\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240084 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81422712-0ebb-4177-a94f-11b2b36bc6ab-kubelet-config\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.240034 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240123 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10357a95-81f6-410a-94db-81289563dcfa-cni-binary-copy\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.240195 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret podName:81422712-0ebb-4177-a94f-11b2b36bc6ab nodeName:}" failed. No retries permitted until 2026-07-16 21:46:47.740139749 +0000 UTC m=+3.073218773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret") pod "global-pull-secret-syncer-5p9fm" (UID: "81422712-0ebb-4177-a94f-11b2b36bc6ab") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240309 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81422712-0ebb-4177-a94f-11b2b36bc6ab-dbus\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240526 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-run-ovn\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240539 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-env-overrides\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240603 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-host-kubelet\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.242955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240657 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-sysctl-conf\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.243717 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.240735 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10357a95-81f6-410a-94db-81289563dcfa-host-var-lib-kubelet\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.243717 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.241001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10357a95-81f6-410a-94db-81289563dcfa-multus-daemon-config\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.243717 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.242267 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/132ee170-dfa0-45fa-a378-11f8a5cc9a76-etc-tuned\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.243717 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.243070 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-ovn-node-metrics-cert\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.243717 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.243551 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/132ee170-dfa0-45fa-a378-11f8a5cc9a76-tmp\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.244418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.244393 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/51fea26c-fe2e-4d70-94c2-03c76aee179c-agent-certs\") pod \"konnectivity-agent-6s8l9\" (UID: \"51fea26c-fe2e-4d70-94c2-03c76aee179c\") " pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:46:47.249856 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.249835 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpflq\" (UniqueName: \"kubernetes.io/projected/132ee170-dfa0-45fa-a378-11f8a5cc9a76-kube-api-access-zpflq\") pod \"tuned-g6cwc\" (UID: \"132ee170-dfa0-45fa-a378-11f8a5cc9a76\") " pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.250493 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.250471 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:46:47.250588 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.250498 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:46:47.250588 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.250512 2563 projected.go:196] Error preparing data for projected volume kube-api-access-w56zs for pod openshift-network-diagnostics/network-check-target-8qd4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:47.250588 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.250574 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs podName:8c0517ca-cafe-4333-9a1c-1eacc489088e nodeName:}" failed. No retries permitted until 2026-07-16 21:46:47.750555158 +0000 UTC m=+3.083634187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w56zs" (UniqueName: "kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs") pod "network-check-target-8qd4b" (UID: "8c0517ca-cafe-4333-9a1c-1eacc489088e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:47.253830 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.253804 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtrf8\" (UniqueName: \"kubernetes.io/projected/e47fb831-76fb-4d1c-a7d1-bf21f1935483-kube-api-access-rtrf8\") pod \"aws-ebs-csi-driver-node-2w8qc\" (UID: \"e47fb831-76fb-4d1c-a7d1-bf21f1935483\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.254600 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.254566 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nml72\" (UniqueName: \"kubernetes.io/projected/c3dbc476-9c43-48c3-a2bf-78a67110c028-kube-api-access-nml72\") pod \"node-resolver-fpm78\" (UID: \"c3dbc476-9c43-48c3-a2bf-78a67110c028\") " pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.254762 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.254736 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqflk\" (UniqueName: \"kubernetes.io/projected/1fd22b1e-b443-499f-b9b0-25bdb51e42e5-kube-api-access-gqflk\") pod \"ovnkube-node-7xmnr\" (UID: \"1fd22b1e-b443-499f-b9b0-25bdb51e42e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.255029 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.255006 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbrl2\" (UniqueName: \"kubernetes.io/projected/10357a95-81f6-410a-94db-81289563dcfa-kube-api-access-hbrl2\") pod \"multus-mjhl2\" (UID: \"10357a95-81f6-410a-94db-81289563dcfa\") " pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.330293 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.330250 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pzmdz" Jul 16 21:46:47.341363 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.341343 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-btfm6" Jul 16 21:46:47.360273 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.360245 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sv64j" Jul 16 21:46:47.369118 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.369087 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mjhl2" Jul 16 21:46:47.375005 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.374987 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fpm78" Jul 16 21:46:47.383269 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.383243 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:46:47.392109 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.392087 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:46:47.402958 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.402932 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" Jul 16 21:46:47.409760 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.409736 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" Jul 16 21:46:47.642703 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.642614 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:47.642849 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.642777 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:47.642849 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.642837 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:46:48.642822116 +0000 UTC m=+3.975901125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:47.743452 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.743424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:47.743641 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.743554 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:47.743641 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.743640 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret podName:81422712-0ebb-4177-a94f-11b2b36bc6ab nodeName:}" failed. No retries permitted until 2026-07-16 21:46:48.74362343 +0000 UTC m=+4.076702444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret") pod "global-pull-secret-syncer-5p9fm" (UID: "81422712-0ebb-4177-a94f-11b2b36bc6ab") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:47.844489 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:47.844451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:47.844692 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.844664 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:46:47.844831 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.844698 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:46:47.844831 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.844713 2563 projected.go:196] Error preparing data for projected volume kube-api-access-w56zs for pod openshift-network-diagnostics/network-check-target-8qd4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:47.844831 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:47.844781 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs podName:8c0517ca-cafe-4333-9a1c-1eacc489088e nodeName:}" failed. No retries permitted until 2026-07-16 21:46:48.844761633 +0000 UTC m=+4.177840658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w56zs" (UniqueName: "kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs") pod "network-check-target-8qd4b" (UID: "8c0517ca-cafe-4333-9a1c-1eacc489088e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:48.093328 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:48.093219 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:41:47 +0000 UTC" deadline="2028-02-22 03:51:15.15367022 +0000 UTC" Jul 16 21:46:48.093328 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:48.093263 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14046h4m27.060411963s" Jul 16 21:46:48.256251 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.256219 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda160cbfd_0e76_4746_9d1a_e442b9dbdff1.slice/crio-05ee0c4d42579b16696f68cfbd97f65247866c7e353ad72177f7f545285739e0 WatchSource:0}: Error finding container 05ee0c4d42579b16696f68cfbd97f65247866c7e353ad72177f7f545285739e0: Status 404 returned error can't find the container with id 05ee0c4d42579b16696f68cfbd97f65247866c7e353ad72177f7f545285739e0 Jul 16 21:46:48.261669 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.261582 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fea26c_fe2e_4d70_94c2_03c76aee179c.slice/crio-495ed7758c0c2e00e954635c5aee38f62b3e92534bfacdfd1a68b07638a6f324 WatchSource:0}: Error finding container 495ed7758c0c2e00e954635c5aee38f62b3e92534bfacdfd1a68b07638a6f324: Status 404 returned error can't find the container with id 495ed7758c0c2e00e954635c5aee38f62b3e92534bfacdfd1a68b07638a6f324 Jul 16 21:46:48.263253 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.263226 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode47fb831_76fb_4d1c_a7d1_bf21f1935483.slice/crio-261b487e3be10d0bfacd90d19a67a37c4869293d7f4bb6c5313bbf1c42e61390 WatchSource:0}: Error finding container 261b487e3be10d0bfacd90d19a67a37c4869293d7f4bb6c5313bbf1c42e61390: Status 404 returned error can't find the container with id 261b487e3be10d0bfacd90d19a67a37c4869293d7f4bb6c5313bbf1c42e61390 Jul 16 21:46:48.263880 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.263851 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b92567_7296_4fce_b493_dd41959095ce.slice/crio-85bc67ff02b6894af9446b98af32235144d9d530a614826fa396ca691b6bdbbf WatchSource:0}: Error finding container 85bc67ff02b6894af9446b98af32235144d9d530a614826fa396ca691b6bdbbf: Status 404 returned error can't find the container with id 85bc67ff02b6894af9446b98af32235144d9d530a614826fa396ca691b6bdbbf Jul 16 21:46:48.265324 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.265222 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod132ee170_dfa0_45fa_a378_11f8a5cc9a76.slice/crio-eb7a06c94eb569eb521fd601aa459c1ef14a7fbf98c8cf3877c7bff5267cf67e WatchSource:0}: Error finding container eb7a06c94eb569eb521fd601aa459c1ef14a7fbf98c8cf3877c7bff5267cf67e: Status 404 returned error can't find the container with id eb7a06c94eb569eb521fd601aa459c1ef14a7fbf98c8cf3877c7bff5267cf67e Jul 16 21:46:48.288644 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.288608 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3dbc476_9c43_48c3_a2bf_78a67110c028.slice/crio-3440e7e0b2d9cbe7ba707994cc4a4b31b06d736bd0064048c6132ce963b2a246 WatchSource:0}: Error finding container 3440e7e0b2d9cbe7ba707994cc4a4b31b06d736bd0064048c6132ce963b2a246: Status 404 returned error can't find the container with id 3440e7e0b2d9cbe7ba707994cc4a4b31b06d736bd0064048c6132ce963b2a246 Jul 16 21:46:48.290450 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.290425 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10357a95_81f6_410a_94db_81289563dcfa.slice/crio-e0246e5f46e8fb120065c6a791dcd5b06811677a5ea1fe3a8ea3e8d334e02d7a WatchSource:0}: Error finding container e0246e5f46e8fb120065c6a791dcd5b06811677a5ea1fe3a8ea3e8d334e02d7a: Status 404 returned error can't find the container with id e0246e5f46e8fb120065c6a791dcd5b06811677a5ea1fe3a8ea3e8d334e02d7a Jul 16 21:46:48.291402 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.291382 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd22b1e_b443_499f_b9b0_25bdb51e42e5.slice/crio-436daa534bfb82cba7f919f215d9ae06a3fe008ee0feaeff3b38b782863734c8 WatchSource:0}: Error finding container 436daa534bfb82cba7f919f215d9ae06a3fe008ee0feaeff3b38b782863734c8: Status 404 returned error can't find the container with id 436daa534bfb82cba7f919f215d9ae06a3fe008ee0feaeff3b38b782863734c8 Jul 16 21:46:48.292657 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:46:48.292626 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa707431_f054_46aa_a35e_2f743c19bdda.slice/crio-7c74a5cd6f1033096cb5a651c6295e4c84b64cb23f929cc87b8e3d34bb78421c WatchSource:0}: Error finding container 7c74a5cd6f1033096cb5a651c6295e4c84b64cb23f929cc87b8e3d34bb78421c: Status 404 returned error can't find the container with id 7c74a5cd6f1033096cb5a651c6295e4c84b64cb23f929cc87b8e3d34bb78421c Jul 16 21:46:48.651802 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:48.651544 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:48.651802 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:48.651697 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:48.652011 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:48.651843 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:46:50.651827733 +0000 UTC m=+5.984906746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:48.753085 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:48.753049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:48.753259 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:48.753184 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:48.753259 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:48.753241 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret podName:81422712-0ebb-4177-a94f-11b2b36bc6ab nodeName:}" failed. No retries permitted until 2026-07-16 21:46:50.753224485 +0000 UTC m=+6.086303497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret") pod "global-pull-secret-syncer-5p9fm" (UID: "81422712-0ebb-4177-a94f-11b2b36bc6ab") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:48.853396 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:48.853359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:48.854060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:48.853568 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:46:48.854060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:48.853594 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:46:48.854060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:48.853608 2563 projected.go:196] Error preparing data for projected volume kube-api-access-w56zs for pod openshift-network-diagnostics/network-check-target-8qd4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:48.854060 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:48.853671 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs podName:8c0517ca-cafe-4333-9a1c-1eacc489088e nodeName:}" failed. No retries permitted until 2026-07-16 21:46:50.853651556 +0000 UTC m=+6.186730573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w56zs" (UniqueName: "kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs") pod "network-check-target-8qd4b" (UID: "8c0517ca-cafe-4333-9a1c-1eacc489088e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:49.094258 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.094124 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:41:47 +0000 UTC" deadline="2028-02-26 09:17:52.51112347 +0000 UTC" Jul 16 21:46:49.094258 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.094205 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14147h31m3.416923461s" Jul 16 21:46:49.237886 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.236922 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:49.237886 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:49.237071 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:46:49.237886 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.237496 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:49.237886 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:49.237595 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:46:49.237886 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.237671 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:49.237886 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:49.237747 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:46:49.248350 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.248181 2563 generic.go:356] "Generic (PLEG): container finished" podID="0a79463808a4d0705f9376bbce08cef7" containerID="614a82d59b1eddb3213524e2c02b9825c525c38f7ef38ef82832a4440ec6aaf4" exitCode=0 Jul 16 21:46:49.248350 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.248290 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" event={"ID":"0a79463808a4d0705f9376bbce08cef7","Type":"ContainerDied","Data":"614a82d59b1eddb3213524e2c02b9825c525c38f7ef38ef82832a4440ec6aaf4"} Jul 16 21:46:49.269655 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.267879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sv64j" event={"ID":"fa707431-f054-46aa-a35e-2f743c19bdda","Type":"ContainerStarted","Data":"7c74a5cd6f1033096cb5a651c6295e4c84b64cb23f929cc87b8e3d34bb78421c"} Jul 16 21:46:49.271392 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.271192 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mjhl2" event={"ID":"10357a95-81f6-410a-94db-81289563dcfa","Type":"ContainerStarted","Data":"e0246e5f46e8fb120065c6a791dcd5b06811677a5ea1fe3a8ea3e8d334e02d7a"} Jul 16 21:46:49.273381 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.273294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" event={"ID":"e47fb831-76fb-4d1c-a7d1-bf21f1935483","Type":"ContainerStarted","Data":"261b487e3be10d0bfacd90d19a67a37c4869293d7f4bb6c5313bbf1c42e61390"} Jul 16 21:46:49.277663 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.277587 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" event={"ID":"132ee170-dfa0-45fa-a378-11f8a5cc9a76","Type":"ContainerStarted","Data":"eb7a06c94eb569eb521fd601aa459c1ef14a7fbf98c8cf3877c7bff5267cf67e"} Jul 16 21:46:49.281108 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.281048 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-btfm6" event={"ID":"54b92567-7296-4fce-b493-dd41959095ce","Type":"ContainerStarted","Data":"85bc67ff02b6894af9446b98af32235144d9d530a614826fa396ca691b6bdbbf"} Jul 16 21:46:49.297413 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.295205 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pzmdz" event={"ID":"a160cbfd-0e76-4746-9d1a-e442b9dbdff1","Type":"ContainerStarted","Data":"05ee0c4d42579b16696f68cfbd97f65247866c7e353ad72177f7f545285739e0"} Jul 16 21:46:49.301638 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.301612 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" event={"ID":"c1faed2aff0ab49640344e2b4b87931f","Type":"ContainerStarted","Data":"f1ce810e569d54a8dff0c39af18e7646457bdbe9da3b837da2b9eeab54767433"} Jul 16 21:46:49.311025 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.310992 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"436daa534bfb82cba7f919f215d9ae06a3fe008ee0feaeff3b38b782863734c8"} Jul 16 21:46:49.313900 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.313871 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fpm78" event={"ID":"c3dbc476-9c43-48c3-a2bf-78a67110c028","Type":"ContainerStarted","Data":"3440e7e0b2d9cbe7ba707994cc4a4b31b06d736bd0064048c6132ce963b2a246"} Jul 16 21:46:49.315555 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.315490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6s8l9" event={"ID":"51fea26c-fe2e-4d70-94c2-03c76aee179c","Type":"ContainerStarted","Data":"495ed7758c0c2e00e954635c5aee38f62b3e92534bfacdfd1a68b07638a6f324"} Jul 16 21:46:49.323249 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:49.322287 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-31.ec2.internal" podStartSLOduration=3.3222700290000002 podStartE2EDuration="3.322270029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:46:49.322145996 +0000 UTC m=+4.655225049" watchObservedRunningTime="2026-07-16 21:46:49.322270029 +0000 UTC m=+4.655349060" Jul 16 21:46:50.338082 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:50.338043 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" event={"ID":"0a79463808a4d0705f9376bbce08cef7","Type":"ContainerStarted","Data":"ef0083cba2ef14cad964b79598f7bc68117b642dfbf9a5feeadb85105e037c64"} Jul 16 21:46:50.667920 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:50.667832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:50.668124 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:50.667985 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:50.668124 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:50.668053 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:46:54.668033585 +0000 UTC m=+10.001112599 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:50.768806 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:50.768768 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:50.768987 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:50.768931 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:50.769080 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:50.769001 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret podName:81422712-0ebb-4177-a94f-11b2b36bc6ab nodeName:}" failed. No retries permitted until 2026-07-16 21:46:54.76898066 +0000 UTC m=+10.102059689 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret") pod "global-pull-secret-syncer-5p9fm" (UID: "81422712-0ebb-4177-a94f-11b2b36bc6ab") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:50.870181 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:50.870137 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:50.870378 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:50.870361 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:46:50.870459 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:50.870389 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:46:50.870459 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:50.870403 2563 projected.go:196] Error preparing data for projected volume kube-api-access-w56zs for pod openshift-network-diagnostics/network-check-target-8qd4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:50.870581 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:50.870485 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs podName:8c0517ca-cafe-4333-9a1c-1eacc489088e nodeName:}" failed. No retries permitted until 2026-07-16 21:46:54.870464857 +0000 UTC m=+10.203543884 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w56zs" (UniqueName: "kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs") pod "network-check-target-8qd4b" (UID: "8c0517ca-cafe-4333-9a1c-1eacc489088e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:51.235424 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:51.235353 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:51.235424 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:51.235379 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:51.235679 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:51.235492 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:46:51.235975 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:51.235808 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:51.235975 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:51.235926 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:46:51.236184 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:51.236048 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:46:53.235086 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:53.234999 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:53.235086 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:53.235040 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:53.235632 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:53.235000 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:53.235632 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:53.235130 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:46:53.235632 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:53.235272 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:46:53.235632 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:53.235366 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:46:54.701968 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:54.701933 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:54.702409 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:54.702093 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:54.702409 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:54.702169 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:02.702151259 +0000 UTC m=+18.035230322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:46:54.803285 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:54.803246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:54.803443 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:54.803375 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:54.803500 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:54.803448 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret podName:81422712-0ebb-4177-a94f-11b2b36bc6ab nodeName:}" failed. No retries permitted until 2026-07-16 21:47:02.80342833 +0000 UTC m=+18.136507349 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret") pod "global-pull-secret-syncer-5p9fm" (UID: "81422712-0ebb-4177-a94f-11b2b36bc6ab") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:46:54.904030 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:54.903978 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:54.904224 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:54.904179 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:46:54.904224 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:54.904203 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:46:54.904224 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:54.904215 2563 projected.go:196] Error preparing data for projected volume kube-api-access-w56zs for pod openshift-network-diagnostics/network-check-target-8qd4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:54.904378 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:54.904280 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs podName:8c0517ca-cafe-4333-9a1c-1eacc489088e nodeName:}" failed. No retries permitted until 2026-07-16 21:47:02.904260241 +0000 UTC m=+18.237339274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w56zs" (UniqueName: "kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs") pod "network-check-target-8qd4b" (UID: "8c0517ca-cafe-4333-9a1c-1eacc489088e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:46:55.236240 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:55.235774 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:55.236240 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:55.235888 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:46:55.236240 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:55.235945 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:55.236240 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:55.236020 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:46:55.236240 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:55.236058 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:55.236240 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:55.236126 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:46:57.236920 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:57.236525 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:57.236920 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:57.236529 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:57.236920 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:57.236649 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:46:57.236920 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:57.236689 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:57.236920 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:57.236788 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:46:57.236920 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:57.236878 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:46:59.234691 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:59.234656 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:46:59.234691 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:59.234688 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:46:59.235185 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:46:59.234657 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:46:59.235185 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:59.234784 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:46:59.235185 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:59.234853 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:46:59.235185 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:46:59.234918 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:01.234327 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:01.234291 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:01.234790 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:01.234291 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:01.234790 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:01.234416 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:01.234790 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:01.234526 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:01.234790 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:01.234289 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:01.234790 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:01.234627 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:02.761407 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:02.761373 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:02.761845 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:02.761507 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:47:02.761845 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:02.761564 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:18.761547063 +0000 UTC m=+34.094626085 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:47:02.862530 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:02.862498 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:02.862692 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:02.862626 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:47:02.862692 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:02.862689 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret podName:81422712-0ebb-4177-a94f-11b2b36bc6ab nodeName:}" failed. No retries permitted until 2026-07-16 21:47:18.862676214 +0000 UTC m=+34.195755222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret") pod "global-pull-secret-syncer-5p9fm" (UID: "81422712-0ebb-4177-a94f-11b2b36bc6ab") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:47:02.963154 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:02.963112 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:02.963321 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:02.963275 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:47:02.963321 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:02.963295 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:47:02.963321 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:02.963304 2563 projected.go:196] Error preparing data for projected volume kube-api-access-w56zs for pod openshift-network-diagnostics/network-check-target-8qd4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:47:02.963454 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:02.963358 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs podName:8c0517ca-cafe-4333-9a1c-1eacc489088e nodeName:}" failed. No retries permitted until 2026-07-16 21:47:18.963343194 +0000 UTC m=+34.296422202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w56zs" (UniqueName: "kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs") pod "network-check-target-8qd4b" (UID: "8c0517ca-cafe-4333-9a1c-1eacc489088e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:47:03.235000 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:03.234968 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:03.235000 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:03.235001 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:03.235222 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:03.234970 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:03.235222 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:03.235100 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:03.235222 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:03.235193 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:03.235356 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:03.235281 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:05.235580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:05.235545 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:05.236023 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:05.235656 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:05.236023 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:05.235764 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:05.236023 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:05.235890 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:05.236023 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:05.235925 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:05.236023 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:05.235999 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:07.235005 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:07.234971 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:07.235453 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:07.234971 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:07.235453 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:07.235077 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:07.235453 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:07.235177 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:07.235453 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:07.235231 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:07.235453 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:07.235323 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:08.377198 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.376716 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mjhl2" event={"ID":"10357a95-81f6-410a-94db-81289563dcfa","Type":"ContainerStarted","Data":"a80f8d8bfb69528e23cd8c35b4f722559e3fdc58b69cb8f9702fef95d2279139"} Jul 16 21:47:08.378216 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.378184 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" event={"ID":"e47fb831-76fb-4d1c-a7d1-bf21f1935483","Type":"ContainerStarted","Data":"be01bef599e104ffa95f740e342c0d7c0e1ff3f42433220b0eabedeccf279585"} Jul 16 21:47:08.379657 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.379634 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" event={"ID":"132ee170-dfa0-45fa-a378-11f8a5cc9a76","Type":"ContainerStarted","Data":"40a9429caf5e3b89bda9f5ed7221ec2dccd992173c73edbb10f2adf3f7986204"} Jul 16 21:47:08.381077 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.381053 2563 generic.go:356] "Generic (PLEG): container finished" podID="54b92567-7296-4fce-b493-dd41959095ce" containerID="0a0d48c206f394d078301817e115b147f6a59f2bf663d9b6a0b3c2c2467cbb75" exitCode=0 Jul 16 21:47:08.381218 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.381124 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-btfm6" event={"ID":"54b92567-7296-4fce-b493-dd41959095ce","Type":"ContainerDied","Data":"0a0d48c206f394d078301817e115b147f6a59f2bf663d9b6a0b3c2c2467cbb75"} Jul 16 21:47:08.382601 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.382581 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pzmdz" event={"ID":"a160cbfd-0e76-4746-9d1a-e442b9dbdff1","Type":"ContainerStarted","Data":"107d02adb0479fc6adb7b273d711eba1316905be0715978eeb9ed5b88e270569"} Jul 16 21:47:08.385266 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.385249 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 21:47:08.385591 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.385572 2563 generic.go:356] "Generic (PLEG): container finished" podID="1fd22b1e-b443-499f-b9b0-25bdb51e42e5" containerID="7cb4a83b9068df2348c5cfcce7d67972fbfbe891c9141533d12dd74d8c3776cf" exitCode=1 Jul 16 21:47:08.385661 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.385633 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"834504411cfd044a0179b5bbcae349110fb536930aca933910b21a167403cb15"} Jul 16 21:47:08.385711 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.385685 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"f2becd9c17f8ba2dd67d1b97dd66dc35fb0cf71db4819f137caaf1db242ffd78"} Jul 16 21:47:08.385753 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.385728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"99720ed4dd72b242797a218166b04e0b6228ae5554411f6b9e862c42e01b10bd"} Jul 16 21:47:08.385791 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.385766 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"280595082bc2b901d6fc6769250dfd5372476e7b9af6374107203b38fe6a9d6a"} Jul 16 21:47:08.385840 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.385810 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerDied","Data":"7cb4a83b9068df2348c5cfcce7d67972fbfbe891c9141533d12dd74d8c3776cf"} Jul 16 21:47:08.385886 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.385850 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"57b69aeea6f97200afc85e648dc0877a4264634bcc96b69f7ae17d0df03b16be"} Jul 16 21:47:08.386936 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.386913 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fpm78" event={"ID":"c3dbc476-9c43-48c3-a2bf-78a67110c028","Type":"ContainerStarted","Data":"a4ca666d58c84d354e7cc1780eb61c2698be0ba7f1b29a0c357fabb619ee5f09"} Jul 16 21:47:08.388279 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.388262 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6s8l9" event={"ID":"51fea26c-fe2e-4d70-94c2-03c76aee179c","Type":"ContainerStarted","Data":"58160228ef255b27a9bfc2428eab68d7a5f9898f706ffde7b26c8617288f80e3"} Jul 16 21:47:08.397333 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.397298 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mjhl2" podStartSLOduration=4.196996281 podStartE2EDuration="23.397270721s" totalImagesPullingTime="19.20027444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:08.397192154 +0000 UTC m=+23.730271197" watchObservedRunningTime="2026-07-16 21:47:08.397270721 +0000 UTC m=+23.730349732" Jul 16 21:47:08.397674 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.397655 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-31.ec2.internal" podStartSLOduration=22.397647648 podStartE2EDuration="22.397647648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:46:50.358816831 +0000 UTC m=+5.691895868" watchObservedRunningTime="2026-07-16 21:47:08.397647648 +0000 UTC m=+23.730726678" Jul 16 21:47:08.419442 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.419407 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g6cwc" podStartSLOduration=3.242211698 podStartE2EDuration="22.419396409s" totalImagesPullingTime="19.177184711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:08.419221577 +0000 UTC m=+23.752300618" watchObservedRunningTime="2026-07-16 21:47:08.419396409 +0000 UTC m=+23.752475439" Jul 16 21:47:08.466365 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.466329 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pzmdz" podStartSLOduration=4.260428493 podStartE2EDuration="23.466319046s" totalImagesPullingTime="19.205890553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:08.466269496 +0000 UTC m=+23.799348526" watchObservedRunningTime="2026-07-16 21:47:08.466319046 +0000 UTC m=+23.799398076" Jul 16 21:47:08.505294 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.505258 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fpm78" podStartSLOduration=3.338564366 podStartE2EDuration="22.505248587s" totalImagesPullingTime="19.166684221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:08.504624354 +0000 UTC m=+23.837703385" watchObservedRunningTime="2026-07-16 21:47:08.505248587 +0000 UTC m=+23.838327616" Jul 16 21:47:08.528409 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:08.528091 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6s8l9" podStartSLOduration=11.496399205 podStartE2EDuration="22.528076244s" totalImagesPullingTime="11.031677039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:08.527654502 +0000 UTC m=+23.860733532" watchObservedRunningTime="2026-07-16 21:47:08.528076244 +0000 UTC m=+23.861155274" Jul 16 21:47:09.234366 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:09.234334 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:09.234366 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:09.234364 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:09.234629 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:09.234334 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:09.234629 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:09.234472 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:09.234629 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:09.234609 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:09.234750 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:09.234678 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:09.393694 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:09.393591 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sv64j" event={"ID":"fa707431-f054-46aa-a35e-2f743c19bdda","Type":"ContainerStarted","Data":"e53f0b9d59797124f5e446ed9be6363cbe33a256eea74f4df0864f281f969617"} Jul 16 21:47:09.415494 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:09.415448 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sv64j" podStartSLOduration=5.248104678 podStartE2EDuration="24.415431054s" totalImagesPullingTime="19.167326376s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:09.41472754 +0000 UTC m=+24.747806595" watchObservedRunningTime="2026-07-16 21:47:09.415431054 +0000 UTC m=+24.748510083" Jul 16 21:47:09.640404 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:09.640377 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 21:47:10.054942 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:10.054712 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:47:10.055282 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:10.055266 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:47:10.100440 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:10.100298 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T21:47:09.640399797Z","UUID":"72d9e8cb-87fe-43e4-be7a-6f87277b84d7","Handler":null,"Name":"","Endpoint":""} Jul 16 21:47:10.101739 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:10.101718 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 21:47:10.101875 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:10.101756 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 21:47:10.397085 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:10.397056 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" event={"ID":"e47fb831-76fb-4d1c-a7d1-bf21f1935483","Type":"ContainerStarted","Data":"c51d78453c6d39c927a82cd26ab91b1faffc9a401a0b7aa36225fdf7efd4b7e6"} Jul 16 21:47:10.397595 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:10.397575 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:47:10.398040 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:10.398022 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6s8l9" Jul 16 21:47:11.234236 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:11.234166 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:11.234236 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:11.234187 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:11.234408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:11.234270 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:11.234408 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:11.234272 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:11.234408 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:11.234363 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:11.234519 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:11.234477 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:11.403571 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:11.403538 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 21:47:11.404229 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:11.403962 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"82de219dab975859c6f761d655a0228b2ca6aee49063f0db75071a27d4897c3c"} Jul 16 21:47:11.406476 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:11.406448 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" event={"ID":"e47fb831-76fb-4d1c-a7d1-bf21f1935483","Type":"ContainerStarted","Data":"7a413d3cd85f636af1c88c52c1ea5f9bf52c35a3fddd434426bbd3c3e471100c"} Jul 16 21:47:11.430047 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:11.429991 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2w8qc" podStartSLOduration=3.028753197 podStartE2EDuration="25.429971142s" totalImagesPullingTime="22.401217945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:11.429203041 +0000 UTC m=+26.762282072" watchObservedRunningTime="2026-07-16 21:47:11.429971142 +0000 UTC m=+26.763050174" Jul 16 21:47:13.234956 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:13.234833 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:13.235357 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:13.234847 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:13.235357 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:13.235038 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:13.235357 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:13.234903 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:13.235357 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:13.235165 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:13.235357 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:13.235237 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:13.415387 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:13.415364 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 21:47:13.415695 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:13.415669 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"4ee3ff9f6957bd7550506970747822a02b349571a65e161b4e03a7f0053747e8"} Jul 16 21:47:13.415935 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:13.415917 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:47:13.416108 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:13.416094 2563 scope.go:117] "RemoveContainer" containerID="7cb4a83b9068df2348c5cfcce7d67972fbfbe891c9141533d12dd74d8c3776cf" Jul 16 21:47:13.431032 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:13.431007 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:47:14.420712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:14.420617 2563 generic.go:356] "Generic (PLEG): container finished" podID="54b92567-7296-4fce-b493-dd41959095ce" containerID="0dee2eba906fe62c9da452440e387333aa6f15412ee2cc1db33274a21678c4f8" exitCode=0 Jul 16 21:47:14.420712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:14.420705 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-btfm6" event={"ID":"54b92567-7296-4fce-b493-dd41959095ce","Type":"ContainerDied","Data":"0dee2eba906fe62c9da452440e387333aa6f15412ee2cc1db33274a21678c4f8"} Jul 16 21:47:14.429755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:14.429718 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 21:47:14.430178 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:14.430135 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" event={"ID":"1fd22b1e-b443-499f-b9b0-25bdb51e42e5","Type":"ContainerStarted","Data":"6ec659803beb4a001a0fb827082128977ef5dcf1b5c1e4623cdf3201e85a386b"} Jul 16 21:47:14.430451 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:14.430426 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:47:14.430568 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:14.430509 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:47:14.447070 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:14.447040 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:47:15.202373 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:15.202327 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" podStartSLOduration=9.841939784000001 podStartE2EDuration="29.202312092s" totalImagesPullingTime="19.360372308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:14.486486867 +0000 UTC m=+29.819565899" watchObservedRunningTime="2026-07-16 21:47:15.202312092 +0000 UTC m=+30.535391125" Jul 16 21:47:15.202598 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:15.202586 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5p9fm"] Jul 16 21:47:15.202704 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:15.202693 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:15.202782 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:15.202767 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:15.205467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:15.205445 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8qd4b"] Jul 16 21:47:15.205566 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:15.205526 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:15.205605 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:15.205587 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:15.208031 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:15.208010 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xm229"] Jul 16 21:47:15.208121 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:15.208108 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:15.208255 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:15.208218 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:16.435745 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:16.435506 2563 generic.go:356] "Generic (PLEG): container finished" podID="54b92567-7296-4fce-b493-dd41959095ce" containerID="cf058e5a09f3838914e78eb8094975a34936cbf6b3d852c104cb6cf570f9ff0b" exitCode=0 Jul 16 21:47:16.435745 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:16.435583 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-btfm6" event={"ID":"54b92567-7296-4fce-b493-dd41959095ce","Type":"ContainerDied","Data":"cf058e5a09f3838914e78eb8094975a34936cbf6b3d852c104cb6cf570f9ff0b"} Jul 16 21:47:17.235194 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:17.235107 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:17.235319 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:17.235107 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:17.235319 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:17.235227 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:17.235319 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:17.235123 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:17.235319 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:17.235305 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:17.235447 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:17.235397 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:18.442270 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:18.442239 2563 generic.go:356] "Generic (PLEG): container finished" podID="54b92567-7296-4fce-b493-dd41959095ce" containerID="9db709f5a46fa872a0e85f443ff6f7bdbc3b92ebb8a95f0019781a157a5e51b7" exitCode=0 Jul 16 21:47:18.442864 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:18.442288 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-btfm6" event={"ID":"54b92567-7296-4fce-b493-dd41959095ce","Type":"ContainerDied","Data":"9db709f5a46fa872a0e85f443ff6f7bdbc3b92ebb8a95f0019781a157a5e51b7"} Jul 16 21:47:18.781291 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:18.781206 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:18.781426 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:18.781337 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:47:18.781426 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:18.781393 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:50.781378729 +0000 UTC m=+66.114457741 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 21:47:18.881628 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:18.881598 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:18.881746 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:18.881710 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 21:47:18.881779 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:18.881753 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret podName:81422712-0ebb-4177-a94f-11b2b36bc6ab nodeName:}" failed. No retries permitted until 2026-07-16 21:47:50.881740534 +0000 UTC m=+66.214819542 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret") pod "global-pull-secret-syncer-5p9fm" (UID: "81422712-0ebb-4177-a94f-11b2b36bc6ab") : object "kube-system"/"original-pull-secret" not registered Jul 16 21:47:18.982403 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:18.982372 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:18.982566 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:18.982539 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 21:47:18.982566 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:18.982566 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 21:47:18.982669 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:18.982576 2563 projected.go:196] Error preparing data for projected volume kube-api-access-w56zs for pod openshift-network-diagnostics/network-check-target-8qd4b: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:47:18.982669 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:18.982626 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs podName:8c0517ca-cafe-4333-9a1c-1eacc489088e nodeName:}" failed. No retries permitted until 2026-07-16 21:47:50.982609862 +0000 UTC m=+66.315688869 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w56zs" (UniqueName: "kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs") pod "network-check-target-8qd4b" (UID: "8c0517ca-cafe-4333-9a1c-1eacc489088e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 21:47:19.234874 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:19.234796 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:19.234874 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:19.234823 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:19.235056 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:19.234806 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:19.235056 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:19.234901 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8qd4b" podUID="8c0517ca-cafe-4333-9a1c-1eacc489088e" Jul 16 21:47:19.235056 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:19.234971 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:47:19.235168 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:19.235066 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5p9fm" podUID="81422712-0ebb-4177-a94f-11b2b36bc6ab" Jul 16 21:47:20.470590 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.470520 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-31.ec2.internal" event="NodeReady" Jul 16 21:47:20.471030 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.470639 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 21:47:20.507372 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.507324 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58f7c6584-lw844"] Jul 16 21:47:20.510755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.510740 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.513753 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.513728 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 21:47:20.513873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.513832 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 21:47:20.513942 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.513929 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 21:47:20.514046 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.514031 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j8r94\"" Jul 16 21:47:20.521066 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.521045 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 21:47:20.522200 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.522180 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vms7h"] Jul 16 21:47:20.525544 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.525520 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.525647 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.525610 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58f7c6584-lw844"] Jul 16 21:47:20.526487 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.526467 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f22kb"] Jul 16 21:47:20.528363 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.528343 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-275zl\"" Jul 16 21:47:20.528667 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.528648 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 21:47:20.528736 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.528670 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 21:47:20.530032 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.530015 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f22kb" Jul 16 21:47:20.532906 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.532866 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 21:47:20.532906 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.532877 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 21:47:20.533038 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.533002 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7z9z2\"" Jul 16 21:47:20.533106 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.533090 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 21:47:20.536785 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.536764 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vms7h"] Jul 16 21:47:20.548910 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.548888 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f22kb"] Jul 16 21:47:20.595731 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.595710 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.595863 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.595738 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkggg\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-kube-api-access-hkggg\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.595863 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.595755 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/963b9f29-793e-4f12-a7bc-426a7b06dd53-config-volume\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.595863 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.595789 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91f0e0b2-de7f-46f9-b428-f0b1e1546653-ca-trust-extracted\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.595863 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.595830 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-trusted-ca\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.596077 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.595916 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-certificates\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.596077 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.595953 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-image-registry-private-configuration\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.596077 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.595978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-bound-sa-token\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.596077 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.596060 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trgxk\" (UniqueName: \"kubernetes.io/projected/963b9f29-793e-4f12-a7bc-426a7b06dd53-kube-api-access-trgxk\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.596295 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.596107 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/963b9f29-793e-4f12-a7bc-426a7b06dd53-tmp-dir\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.596295 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.596157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-installation-pull-secrets\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.596295 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.596177 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.680523 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.680493 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6s8l9_51fea26c-fe2e-4d70-94c2-03c76aee179c/konnectivity-agent/0.log" limit=1024 Jul 16 21:47:20.696518 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.696486 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.696670 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.696525 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkggg\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-kube-api-access-hkggg\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.696670 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.696546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/963b9f29-793e-4f12-a7bc-426a7b06dd53-config-volume\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.696670 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.696574 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91f0e0b2-de7f-46f9-b428-f0b1e1546653-ca-trust-extracted\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.696670 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.696598 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-trusted-ca\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.696670 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:20.696652 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:47:20.696670 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:20.696672 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:47:20.696950 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:20.696732 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:21.196712132 +0000 UTC m=+36.529791160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:47:20.696950 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.696908 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-certificates\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.697050 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.696958 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-image-registry-private-configuration\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.697050 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.696996 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-bound-sa-token\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.697050 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91f0e0b2-de7f-46f9-b428-f0b1e1546653-ca-trust-extracted\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.697050 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697027 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trgxk\" (UniqueName: \"kubernetes.io/projected/963b9f29-793e-4f12-a7bc-426a7b06dd53-kube-api-access-trgxk\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.697275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697055 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzrdh\" (UniqueName: \"kubernetes.io/projected/d366d608-8179-4e70-8529-6dce72b0b665-kube-api-access-qzrdh\") pod \"ingress-canary-f22kb\" (UID: \"d366d608-8179-4e70-8529-6dce72b0b665\") " pod="openshift-ingress-canary/ingress-canary-f22kb" Jul 16 21:47:20.697275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697083 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d366d608-8179-4e70-8529-6dce72b0b665-cert\") pod \"ingress-canary-f22kb\" (UID: \"d366d608-8179-4e70-8529-6dce72b0b665\") " pod="openshift-ingress-canary/ingress-canary-f22kb" Jul 16 21:47:20.697275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697113 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/963b9f29-793e-4f12-a7bc-426a7b06dd53-tmp-dir\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.697275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697137 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-installation-pull-secrets\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.697275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697180 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.697275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697241 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/963b9f29-793e-4f12-a7bc-426a7b06dd53-config-volume\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.697560 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:20.697282 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:47:20.697560 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:20.697322 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:21.197309414 +0000 UTC m=+36.530388422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:47:20.697560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697510 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-certificates\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.697560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697537 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/963b9f29-793e-4f12-a7bc-426a7b06dd53-tmp-dir\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.697751 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.697589 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-trusted-ca\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.701306 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.701284 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-installation-pull-secrets\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.701418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.701284 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-image-registry-private-configuration\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.709308 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.709288 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkggg\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-kube-api-access-hkggg\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.710608 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.710587 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trgxk\" (UniqueName: \"kubernetes.io/projected/963b9f29-793e-4f12-a7bc-426a7b06dd53-kube-api-access-trgxk\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:20.710608 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.710602 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-bound-sa-token\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:20.777170 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.777086 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6s8l9_51fea26c-fe2e-4d70-94c2-03c76aee179c/konnectivity-agent/0.log" limit=1024 Jul 16 21:47:20.798084 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.798057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzrdh\" (UniqueName: \"kubernetes.io/projected/d366d608-8179-4e70-8529-6dce72b0b665-kube-api-access-qzrdh\") pod \"ingress-canary-f22kb\" (UID: \"d366d608-8179-4e70-8529-6dce72b0b665\") " pod="openshift-ingress-canary/ingress-canary-f22kb" Jul 16 21:47:20.798224 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.798087 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d366d608-8179-4e70-8529-6dce72b0b665-cert\") pod \"ingress-canary-f22kb\" (UID: \"d366d608-8179-4e70-8529-6dce72b0b665\") " pod="openshift-ingress-canary/ingress-canary-f22kb" Jul 16 21:47:20.800382 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.800362 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d366d608-8179-4e70-8529-6dce72b0b665-cert\") pod \"ingress-canary-f22kb\" (UID: \"d366d608-8179-4e70-8529-6dce72b0b665\") " pod="openshift-ingress-canary/ingress-canary-f22kb" Jul 16 21:47:20.807606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.807588 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzrdh\" (UniqueName: \"kubernetes.io/projected/d366d608-8179-4e70-8529-6dce72b0b665-kube-api-access-qzrdh\") pod \"ingress-canary-f22kb\" (UID: \"d366d608-8179-4e70-8529-6dce72b0b665\") " pod="openshift-ingress-canary/ingress-canary-f22kb" Jul 16 21:47:20.840853 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.840829 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f22kb" Jul 16 21:47:20.995460 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:20.995189 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f22kb"] Jul 16 21:47:21.000775 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:47:21.000750 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd366d608_8179_4e70_8529_6dce72b0b665.slice/crio-f138fdd64bdb183729d94bdfb1530a99bcf1eae3dae4ff6cf0bed476f931c478 WatchSource:0}: Error finding container f138fdd64bdb183729d94bdfb1530a99bcf1eae3dae4ff6cf0bed476f931c478: Status 404 returned error can't find the container with id f138fdd64bdb183729d94bdfb1530a99bcf1eae3dae4ff6cf0bed476f931c478 Jul 16 21:47:21.201667 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.201555 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:21.201667 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.201597 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:21.201866 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:21.201717 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:47:21.201866 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:21.201784 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:22.201765644 +0000 UTC m=+37.534844669 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:47:21.201866 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:21.201785 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:47:21.201866 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:21.201805 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:47:21.201866 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:21.201855 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:22.201838497 +0000 UTC m=+37.534917522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:47:21.235211 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.235184 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:21.235379 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.235243 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:21.235379 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.235245 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:21.238925 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.238904 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 21:47:21.239069 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.238943 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 21:47:21.239303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.239231 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 21:47:21.239422 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.239317 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 21:47:21.239422 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.239322 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gkkkc\"" Jul 16 21:47:21.239527 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.239428 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wc4rs\"" Jul 16 21:47:21.450328 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:21.450295 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f22kb" event={"ID":"d366d608-8179-4e70-8529-6dce72b0b665","Type":"ContainerStarted","Data":"f138fdd64bdb183729d94bdfb1530a99bcf1eae3dae4ff6cf0bed476f931c478"} Jul 16 21:47:22.208551 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:22.208518 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:22.209059 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:22.208684 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:47:22.209059 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:22.208741 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:24.208725905 +0000 UTC m=+39.541804914 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:47:22.209059 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:22.208686 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:22.209059 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:22.208772 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:47:22.209059 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:22.208787 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:47:22.209059 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:22.208837 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:24.208821676 +0000 UTC m=+39.541900715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:47:24.224336 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:24.224296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:24.224795 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:24.224364 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:24.224795 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:24.224468 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:47:24.224795 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:24.224487 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:47:24.224795 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:24.224502 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:47:24.224795 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:24.224547 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:28.224528558 +0000 UTC m=+43.557607587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:47:24.224795 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:24.224617 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:28.224599716 +0000 UTC m=+43.557678728 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:47:25.463492 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:25.463407 2563 generic.go:356] "Generic (PLEG): container finished" podID="54b92567-7296-4fce-b493-dd41959095ce" containerID="f700027aab4d859847f3f4f88960eaf898ee4fb353de626f7bda30ccf8729aed" exitCode=0 Jul 16 21:47:25.463891 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:25.463492 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-btfm6" event={"ID":"54b92567-7296-4fce-b493-dd41959095ce","Type":"ContainerDied","Data":"f700027aab4d859847f3f4f88960eaf898ee4fb353de626f7bda30ccf8729aed"} Jul 16 21:47:25.464894 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:25.464867 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f22kb" event={"ID":"d366d608-8179-4e70-8529-6dce72b0b665","Type":"ContainerStarted","Data":"3f9cf208fd69d9db72af9932312b701ae190ac3a0837d8af0b44e7379fa06a47"} Jul 16 21:47:25.505888 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:25.505828 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f22kb" podStartSLOduration=1.618087893 podStartE2EDuration="5.50580774s" totalImagesPullingTime="3.887719847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:25.505577863 +0000 UTC m=+40.838656893" watchObservedRunningTime="2026-07-16 21:47:25.50580774 +0000 UTC m=+40.838886771" Jul 16 21:47:26.470398 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:26.470366 2563 generic.go:356] "Generic (PLEG): container finished" podID="54b92567-7296-4fce-b493-dd41959095ce" containerID="6e6955e28843fc7f622a1d3f465a994eac776fc220b7432548eb918017f487a4" exitCode=0 Jul 16 21:47:26.470827 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:26.470450 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-btfm6" event={"ID":"54b92567-7296-4fce-b493-dd41959095ce","Type":"ContainerDied","Data":"6e6955e28843fc7f622a1d3f465a994eac776fc220b7432548eb918017f487a4"} Jul 16 21:47:27.475525 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:27.475491 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-btfm6" event={"ID":"54b92567-7296-4fce-b493-dd41959095ce","Type":"ContainerStarted","Data":"af8914177a29852385ba7d4804942b7e1a790477066980c791fb00540cfaa4bc"} Jul 16 21:47:27.502980 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:27.502933 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-btfm6" podStartSLOduration=9.130538462 podStartE2EDuration="42.502919396s" totalImagesPullingTime="33.372380934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:27.50103732 +0000 UTC m=+42.834116350" watchObservedRunningTime="2026-07-16 21:47:27.502919396 +0000 UTC m=+42.835998425" Jul 16 21:47:28.256898 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:28.256853 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:28.257075 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:28.256915 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:28.257075 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:28.257007 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:47:28.257075 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:28.257057 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:47:28.257075 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:28.257071 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:47:28.257075 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:28.257075 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:36.2570591 +0000 UTC m=+51.590138110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:47:28.257304 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:28.257122 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:36.257107118 +0000 UTC m=+51.590186147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:47:36.316892 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:36.316858 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:36.317338 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:36.316955 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:36.317338 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:36.317018 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:47:36.317338 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:36.317035 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:47:36.317338 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:36.317044 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:47:36.317338 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:36.317097 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:52.317079707 +0000 UTC m=+67.650158733 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:47:36.317338 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:36.317114 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:47:52.317105723 +0000 UTC m=+67.650184733 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:47:46.455105 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:46.455077 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7xmnr" Jul 16 21:47:50.824634 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:50.824602 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:47:50.827785 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:50.827760 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 21:47:50.835554 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:50.835537 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 21:47:50.835633 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:50.835595 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:54.835581192 +0000 UTC m=+130.168660200 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : secret "metrics-daemon-secret" not found Jul 16 21:47:50.925078 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:50.925047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:50.928373 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:50.928357 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 21:47:50.939121 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:50.939098 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81422712-0ebb-4177-a94f-11b2b36bc6ab-original-pull-secret\") pod \"global-pull-secret-syncer-5p9fm\" (UID: \"81422712-0ebb-4177-a94f-11b2b36bc6ab\") " pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:50.954873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:50.954851 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5p9fm" Jul 16 21:47:51.025477 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.025442 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:51.029960 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.029783 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 21:47:51.041317 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.041294 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 21:47:51.049034 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.049009 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w56zs\" (UniqueName: \"kubernetes.io/projected/8c0517ca-cafe-4333-9a1c-1eacc489088e-kube-api-access-w56zs\") pod \"network-check-target-8qd4b\" (UID: \"8c0517ca-cafe-4333-9a1c-1eacc489088e\") " pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:51.085306 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.085118 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5p9fm"] Jul 16 21:47:51.087809 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:47:51.087785 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81422712_0ebb_4177_a94f_11b2b36bc6ab.slice/crio-35ff223dd48ff38da11a8cf94a5cef13d3560211ecb836c300c9c8cbaeff0b03 WatchSource:0}: Error finding container 35ff223dd48ff38da11a8cf94a5cef13d3560211ecb836c300c9c8cbaeff0b03: Status 404 returned error can't find the container with id 35ff223dd48ff38da11a8cf94a5cef13d3560211ecb836c300c9c8cbaeff0b03 Jul 16 21:47:51.264156 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.264118 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wc4rs\"" Jul 16 21:47:51.271639 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.271619 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:51.386429 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.386315 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8qd4b"] Jul 16 21:47:51.388625 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:47:51.388604 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0517ca_cafe_4333_9a1c_1eacc489088e.slice/crio-ef6f5a8a9dd3dbfcddf83955231423356921b9757df06e0a50c640bb0d91a3f6 WatchSource:0}: Error finding container ef6f5a8a9dd3dbfcddf83955231423356921b9757df06e0a50c640bb0d91a3f6: Status 404 returned error can't find the container with id ef6f5a8a9dd3dbfcddf83955231423356921b9757df06e0a50c640bb0d91a3f6 Jul 16 21:47:51.527104 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.527072 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8qd4b" event={"ID":"8c0517ca-cafe-4333-9a1c-1eacc489088e","Type":"ContainerStarted","Data":"ef6f5a8a9dd3dbfcddf83955231423356921b9757df06e0a50c640bb0d91a3f6"} Jul 16 21:47:51.528161 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:51.528120 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5p9fm" event={"ID":"81422712-0ebb-4177-a94f-11b2b36bc6ab","Type":"ContainerStarted","Data":"35ff223dd48ff38da11a8cf94a5cef13d3560211ecb836c300c9c8cbaeff0b03"} Jul 16 21:47:52.334660 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:52.334620 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:47:52.335163 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:52.334693 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:47:52.335163 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:52.334755 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:47:52.335163 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:52.334803 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:47:52.335163 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:52.334815 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:47:52.335163 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:52.334826 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:24.334805627 +0000 UTC m=+99.667884650 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:47:52.335163 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:47:52.334860 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:48:24.334850135 +0000 UTC m=+99.667929158 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:47:56.543034 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:56.542937 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8qd4b" event={"ID":"8c0517ca-cafe-4333-9a1c-1eacc489088e","Type":"ContainerStarted","Data":"ea2d1850c871830c7746568a5426b77d8a9694424b7b4abef848a08f216ad125"} Jul 16 21:47:56.543034 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:56.543034 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:47:56.544334 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:56.544308 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5p9fm" event={"ID":"81422712-0ebb-4177-a94f-11b2b36bc6ab","Type":"ContainerStarted","Data":"fb1fd18a70d6acc6863515e2d0f28ed7a40c3a6b5ebcb7eb60e29c7205c2a284"} Jul 16 21:47:56.564631 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:56.564586 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8qd4b" podStartSLOduration=65.70942525 podStartE2EDuration="1m10.564573231s" totalImagesPullingTime="4.855147981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:56.564081804 +0000 UTC m=+71.897160835" watchObservedRunningTime="2026-07-16 21:47:56.564573231 +0000 UTC m=+71.897652302" Jul 16 21:47:56.584576 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:47:56.584537 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5p9fm" podStartSLOduration=65.427645556 podStartE2EDuration="1m10.584524282s" totalImagesPullingTime="5.156878726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:56.583514156 +0000 UTC m=+71.916593185" watchObservedRunningTime="2026-07-16 21:47:56.584524282 +0000 UTC m=+71.917603311" Jul 16 21:48:24.357434 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:48:24.357288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:48:24.357434 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:48:24.357352 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:48:24.357975 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:48:24.357486 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:48:24.357975 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:48:24.357537 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:48:24.357975 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:48:24.357559 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:48:24.357975 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:48:24.357565 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:28.357542934 +0000 UTC m=+163.690621942 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:48:24.357975 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:48:24.357650 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:49:28.357609276 +0000 UTC m=+163.690688305 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:48:27.548879 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:48:27.548846 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8qd4b" Jul 16 21:48:54.866679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:48:54.866625 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:48:54.867189 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:48:54.866779 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 21:48:54.867189 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:48:54.866889 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:50:56.866870557 +0000 UTC m=+252.199949569 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : secret "metrics-daemon-secret" not found Jul 16 21:49:23.520889 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:49:23.520852 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-58f7c6584-lw844" podUID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" Jul 16 21:49:23.535019 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:49:23.534989 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vms7h" podUID="963b9f29-793e-4f12-a7bc-426a7b06dd53" Jul 16 21:49:23.738775 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:49:23.738745 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vms7h" Jul 16 21:49:23.738929 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:49:23.738745 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:49:24.248533 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:49:24.248494 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:49:28.383423 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:49:28.383366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:49:28.383423 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:49:28.383418 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:49:28.383888 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:49:28.383517 2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 21:49:28.383888 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:49:28.383528 2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58f7c6584-lw844: secret "image-registry-tls" not found Jul 16 21:49:28.383888 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:49:28.383523 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 21:49:28.383888 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:49:28.383581 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls podName:91f0e0b2-de7f-46f9-b428-f0b1e1546653 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:30.383565758 +0000 UTC m=+285.716644766 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls") pod "image-registry-58f7c6584-lw844" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653") : secret "image-registry-tls" not found Jul 16 21:49:28.383888 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:49:28.383604 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls podName:963b9f29-793e-4f12-a7bc-426a7b06dd53 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:30.383586438 +0000 UTC m=+285.716665447 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls") pod "dns-default-vms7h" (UID: "963b9f29-793e-4f12-a7bc-426a7b06dd53") : secret "dns-default-metrics-tls" not found Jul 16 21:49:35.235161 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:49:35.235119 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:50:56.876130 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:50:56.876083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:50:56.876736 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:50:56.876259 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 21:50:56.876736 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:50:56.876345 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs podName:ef310c53-4c7a-4331-91b9-6e7a2bb27841 nodeName:}" failed. No retries permitted until 2026-07-16 21:52:58.876321202 +0000 UTC m=+374.209400217 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs") pod "network-metrics-daemon-xm229" (UID: "ef310c53-4c7a-4331-91b9-6e7a2bb27841") : secret "metrics-daemon-secret" not found Jul 16 21:51:09.032535 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.032458 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw"] Jul 16 21:51:09.035224 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.035206 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:09.040024 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.039949 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 21:51:09.043434 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.043115 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:51:09.044076 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.043705 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 21:51:09.044076 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.043902 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tgxzd\"" Jul 16 21:51:09.045934 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.045908 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-8vbpp"] Jul 16 21:51:09.049181 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.049161 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9"] Jul 16 21:51:09.049274 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.049260 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.051689 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.051672 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-prwdk"] Jul 16 21:51:09.051814 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.051799 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.052890 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.052873 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 21:51:09.053305 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.053289 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:51:09.054038 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.054018 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 21:51:09.054185 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.054074 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 21:51:09.054185 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.054097 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-sdhnh\"" Jul 16 21:51:09.054338 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.054321 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.057313 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.057294 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 21:51:09.057610 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.057319 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-8t9ml\"" Jul 16 21:51:09.057610 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.057445 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 21:51:09.057610 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.057458 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 21:51:09.058290 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.058274 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 21:51:09.060489 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.060381 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 21:51:09.060489 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.060473 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-l97xg\"" Jul 16 21:51:09.060941 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.060913 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw"] Jul 16 21:51:09.061571 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.061537 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 21:51:09.062010 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.061827 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 21:51:09.062010 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.061937 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 21:51:09.064078 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.064052 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 21:51:09.065071 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.065042 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-8vbpp"] Jul 16 21:51:09.065794 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.065776 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9"] Jul 16 21:51:09.066792 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.066771 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-prwdk"] Jul 16 21:51:09.067596 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.067579 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 21:51:09.132046 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.132022 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth"] Jul 16 21:51:09.134913 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.134895 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-544c55cfbb-t62g9"] Jul 16 21:51:09.135035 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.135017 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.137369 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.137353 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.139935 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.139918 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 21:51:09.143180 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143159 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 21:51:09.143414 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143396 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:51:09.143528 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143399 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 21:51:09.143712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143536 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 21:51:09.143712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143568 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2s47x\"" Jul 16 21:51:09.143712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143463 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 21:51:09.143712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143463 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 21:51:09.143712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143478 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 21:51:09.143712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143707 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hdkbt\"" Jul 16 21:51:09.143712 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143703 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 21:51:09.144105 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.143956 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 21:51:09.152249 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152231 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d91210e8-ef77-43ac-8287-c51e70dddbd9-trusted-ca\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.152339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152266 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/81e44120-3c18-4e07-ba65-7ce68c3255e8-snapshots\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.152339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152282 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rqn8\" (UniqueName: \"kubernetes.io/projected/7c827cc4-1d7f-4955-a2e0-270c3f522dab-kube-api-access-5rqn8\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:09.152339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152302 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d91210e8-ef77-43ac-8287-c51e70dddbd9-config\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.152339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81e44120-3c18-4e07-ba65-7ce68c3255e8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.152339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81e44120-3c18-4e07-ba65-7ce68c3255e8-service-ca-bundle\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.152574 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152460 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d91210e8-ef77-43ac-8287-c51e70dddbd9-serving-cert\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.152574 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152500 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81e44120-3c18-4e07-ba65-7ce68c3255e8-tmp\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.152574 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152530 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.152574 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152557 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.152772 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152597 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2z6w\" (UniqueName: \"kubernetes.io/projected/81e44120-3c18-4e07-ba65-7ce68c3255e8-kube-api-access-z2z6w\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.152772 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152648 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81e44120-3c18-4e07-ba65-7ce68c3255e8-serving-cert\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.152772 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152675 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:09.152772 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152708 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8txk8\" (UniqueName: \"kubernetes.io/projected/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-kube-api-access-8txk8\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.152772 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.152767 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqnm6\" (UniqueName: \"kubernetes.io/projected/d91210e8-ef77-43ac-8287-c51e70dddbd9-kube-api-access-kqnm6\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.168065 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.168038 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-544c55cfbb-t62g9"] Jul 16 21:51:09.168819 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.168802 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth"] Jul 16 21:51:09.234134 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.234108 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z"] Jul 16 21:51:09.238511 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.238487 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z" Jul 16 21:51:09.243691 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.243674 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-zsxx2\"" Jul 16 21:51:09.253862 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253841 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5nq6\" (UniqueName: \"kubernetes.io/projected/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-kube-api-access-q5nq6\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.253939 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253873 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d91210e8-ef77-43ac-8287-c51e70dddbd9-serving-cert\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.253939 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253892 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81e44120-3c18-4e07-ba65-7ce68c3255e8-tmp\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.253939 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253908 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-stats-auth\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.253939 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253925 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.254171 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253942 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.254171 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253958 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2z6w\" (UniqueName: \"kubernetes.io/projected/81e44120-3c18-4e07-ba65-7ce68c3255e8-kube-api-access-z2z6w\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.254171 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253976 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81e44120-3c18-4e07-ba65-7ce68c3255e8-serving-cert\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.254171 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.253991 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-default-certificate\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.254171 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254006 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:09.254171 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.254057 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:09.254171 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.254129 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls podName:25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa nodeName:}" failed. No retries permitted until 2026-07-16 21:51:09.754110842 +0000 UTC m=+265.087190073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-64kl9" (UID: "25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254172 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txk8\" (UniqueName: \"kubernetes.io/projected/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-kube-api-access-8txk8\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254201 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254297 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqnm6\" (UniqueName: \"kubernetes.io/projected/d91210e8-ef77-43ac-8287-c51e70dddbd9-kube-api-access-kqnm6\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254347 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81e44120-3c18-4e07-ba65-7ce68c3255e8-tmp\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254350 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d91210e8-ef77-43ac-8287-c51e70dddbd9-trusted-ca\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254424 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/81e44120-3c18-4e07-ba65-7ce68c3255e8-snapshots\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254454 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rqn8\" (UniqueName: \"kubernetes.io/projected/7c827cc4-1d7f-4955-a2e0-270c3f522dab-kube-api-access-5rqn8\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.254356 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254485 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d91210e8-ef77-43ac-8287-c51e70dddbd9-config\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254510 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81e44120-3c18-4e07-ba65-7ce68c3255e8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.254522 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls podName:7c827cc4-1d7f-4955-a2e0-270c3f522dab nodeName:}" failed. No retries permitted until 2026-07-16 21:51:09.754506417 +0000 UTC m=+265.087585426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-bs7nw" (UID: "7c827cc4-1d7f-4955-a2e0-270c3f522dab") : secret "samples-operator-tls" not found Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254555 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.254581 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254584 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lglc8\" (UniqueName: \"kubernetes.io/projected/838fa3a2-b821-45b7-9678-a60fa9e797f2-kube-api-access-lglc8\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.255248 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254609 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-config\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.255248 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254637 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.255248 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254662 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81e44120-3c18-4e07-ba65-7ce68c3255e8-service-ca-bundle\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.255248 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.254852 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.255248 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.255169 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d91210e8-ef77-43ac-8287-c51e70dddbd9-config\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.255443 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.255265 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/81e44120-3c18-4e07-ba65-7ce68c3255e8-snapshots\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.255443 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.255335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d91210e8-ef77-43ac-8287-c51e70dddbd9-trusted-ca\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.255569 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.255553 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81e44120-3c18-4e07-ba65-7ce68c3255e8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.255629 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.255610 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/81e44120-3c18-4e07-ba65-7ce68c3255e8-service-ca-bundle\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.256498 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.256474 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d91210e8-ef77-43ac-8287-c51e70dddbd9-serving-cert\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.256586 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.256555 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/81e44120-3c18-4e07-ba65-7ce68c3255e8-serving-cert\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.260570 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.260549 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z"] Jul 16 21:51:09.275853 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.275816 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqnm6\" (UniqueName: \"kubernetes.io/projected/d91210e8-ef77-43ac-8287-c51e70dddbd9-kube-api-access-kqnm6\") pod \"console-operator-565b7b569d-8vbpp\" (UID: \"d91210e8-ef77-43ac-8287-c51e70dddbd9\") " pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.280885 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.280866 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8txk8\" (UniqueName: \"kubernetes.io/projected/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-kube-api-access-8txk8\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.288927 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.288881 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rqn8\" (UniqueName: \"kubernetes.io/projected/7c827cc4-1d7f-4955-a2e0-270c3f522dab-kube-api-access-5rqn8\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:09.315135 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.315117 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2z6w\" (UniqueName: \"kubernetes.io/projected/81e44120-3c18-4e07-ba65-7ce68c3255e8-kube-api-access-z2z6w\") pod \"insights-operator-555d9d7885-prwdk\" (UID: \"81e44120-3c18-4e07-ba65-7ce68c3255e8\") " pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.355357 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355339 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.355457 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lglc8\" (UniqueName: \"kubernetes.io/projected/838fa3a2-b821-45b7-9678-a60fa9e797f2-kube-api-access-lglc8\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.355457 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355383 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-config\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.355457 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355398 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.355457 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355434 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5nq6\" (UniqueName: \"kubernetes.io/projected/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-kube-api-access-q5nq6\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.355713 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355461 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-stats-auth\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.355713 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355506 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrhhv\" (UniqueName: \"kubernetes.io/projected/e7ee6d63-b06e-4a91-a0c4-f86e7f25b3d4-kube-api-access-qrhhv\") pod \"network-check-source-7bc5b6bf85-97n2z\" (UID: \"e7ee6d63-b06e-4a91-a0c4-f86e7f25b3d4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z" Jul 16 21:51:09.355713 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355540 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-default-certificate\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.355713 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355626 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.355713 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.355637 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:09.855619597 +0000 UTC m=+265.188698606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : configmap references non-existent config key: service-ca.crt Jul 16 21:51:09.355713 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.355694 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:51:09.356020 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.355753 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:09.855736539 +0000 UTC m=+265.188815547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : secret "router-metrics-certs-default" not found Jul 16 21:51:09.356020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.355975 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-config\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.357360 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.357339 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-serving-cert\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.357703 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.357687 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-stats-auth\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.357756 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.357741 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-default-certificate\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.358572 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.358559 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:09.365442 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.365423 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lglc8\" (UniqueName: \"kubernetes.io/projected/838fa3a2-b821-45b7-9678-a60fa9e797f2-kube-api-access-lglc8\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.372709 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.372693 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-prwdk" Jul 16 21:51:09.374427 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.374402 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5nq6\" (UniqueName: \"kubernetes.io/projected/b738ba0e-3834-4ff6-9f6b-15d5f79a132d-kube-api-access-q5nq6\") pod \"service-ca-operator-5bcbc94c78-v9hth\" (UID: \"b738ba0e-3834-4ff6-9f6b-15d5f79a132d\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.444962 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.444931 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" Jul 16 21:51:09.456710 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.456608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrhhv\" (UniqueName: \"kubernetes.io/projected/e7ee6d63-b06e-4a91-a0c4-f86e7f25b3d4-kube-api-access-qrhhv\") pod \"network-check-source-7bc5b6bf85-97n2z\" (UID: \"e7ee6d63-b06e-4a91-a0c4-f86e7f25b3d4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z" Jul 16 21:51:09.467080 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.467056 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrhhv\" (UniqueName: \"kubernetes.io/projected/e7ee6d63-b06e-4a91-a0c4-f86e7f25b3d4-kube-api-access-qrhhv\") pod \"network-check-source-7bc5b6bf85-97n2z\" (UID: \"e7ee6d63-b06e-4a91-a0c4-f86e7f25b3d4\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z" Jul 16 21:51:09.488836 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.488810 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-8vbpp"] Jul 16 21:51:09.491106 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:09.491067 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91210e8_ef77_43ac_8287_c51e70dddbd9.slice/crio-c035ef8e84ae235067d36099812431a87b988ea8f86877e28a7287d0d3f1e5dc WatchSource:0}: Error finding container c035ef8e84ae235067d36099812431a87b988ea8f86877e28a7287d0d3f1e5dc: Status 404 returned error can't find the container with id c035ef8e84ae235067d36099812431a87b988ea8f86877e28a7287d0d3f1e5dc Jul 16 21:51:09.506378 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.506354 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-prwdk"] Jul 16 21:51:09.508363 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:09.508337 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e44120_3c18_4e07_ba65_7ce68c3255e8.slice/crio-f448543b228f97cf74df9b0aef24430f57cbf7c01e156ab534a36c7526de407d WatchSource:0}: Error finding container f448543b228f97cf74df9b0aef24430f57cbf7c01e156ab534a36c7526de407d: Status 404 returned error can't find the container with id f448543b228f97cf74df9b0aef24430f57cbf7c01e156ab534a36c7526de407d Jul 16 21:51:09.547600 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.547538 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z" Jul 16 21:51:09.571357 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.571332 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth"] Jul 16 21:51:09.575023 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:09.574986 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb738ba0e_3834_4ff6_9f6b_15d5f79a132d.slice/crio-e359e7ec5b88ca1c8af14b4de980a028635412eb5ef7e100c7021296cd3fdbe3 WatchSource:0}: Error finding container e359e7ec5b88ca1c8af14b4de980a028635412eb5ef7e100c7021296cd3fdbe3: Status 404 returned error can't find the container with id e359e7ec5b88ca1c8af14b4de980a028635412eb5ef7e100c7021296cd3fdbe3 Jul 16 21:51:09.667804 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.667783 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z"] Jul 16 21:51:09.669420 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:09.669397 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7ee6d63_b06e_4a91_a0c4_f86e7f25b3d4.slice/crio-914ec6c9d785f2a48d3b8904406cd08ea774d5db2c5bb2986d8b8bd5c16a3357 WatchSource:0}: Error finding container 914ec6c9d785f2a48d3b8904406cd08ea774d5db2c5bb2986d8b8bd5c16a3357: Status 404 returned error can't find the container with id 914ec6c9d785f2a48d3b8904406cd08ea774d5db2c5bb2986d8b8bd5c16a3357 Jul 16 21:51:09.758617 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.758588 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:09.758758 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.758643 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:09.758758 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.758737 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:09.758865 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.758779 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:51:09.758865 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.758795 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls podName:25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa nodeName:}" failed. No retries permitted until 2026-07-16 21:51:10.758776848 +0000 UTC m=+266.091855873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-64kl9" (UID: "25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:09.758865 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.758836 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls podName:7c827cc4-1d7f-4955-a2e0-270c3f522dab nodeName:}" failed. No retries permitted until 2026-07-16 21:51:10.758818633 +0000 UTC m=+266.091897648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-bs7nw" (UID: "7c827cc4-1d7f-4955-a2e0-270c3f522dab") : secret "samples-operator-tls" not found Jul 16 21:51:09.859295 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.859268 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.859423 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.859319 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:09.859423 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.859397 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:51:09.859504 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.859407 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:10.859395487 +0000 UTC m=+266.192474495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : configmap references non-existent config key: service-ca.crt Jul 16 21:51:09.859504 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:09.859454 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:10.859441913 +0000 UTC m=+266.192520920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : secret "router-metrics-certs-default" not found Jul 16 21:51:09.988422 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.988354 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" event={"ID":"b738ba0e-3834-4ff6-9f6b-15d5f79a132d","Type":"ContainerStarted","Data":"e359e7ec5b88ca1c8af14b4de980a028635412eb5ef7e100c7021296cd3fdbe3"} Jul 16 21:51:09.989543 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.989505 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" event={"ID":"d91210e8-ef77-43ac-8287-c51e70dddbd9","Type":"ContainerStarted","Data":"c035ef8e84ae235067d36099812431a87b988ea8f86877e28a7287d0d3f1e5dc"} Jul 16 21:51:09.991333 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.991064 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z" event={"ID":"e7ee6d63-b06e-4a91-a0c4-f86e7f25b3d4","Type":"ContainerStarted","Data":"ed5117f96a0633f7198f0cd23a91b178954a939eb7f08e548ff060a8c5e9e7b3"} Jul 16 21:51:09.991333 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.991107 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z" event={"ID":"e7ee6d63-b06e-4a91-a0c4-f86e7f25b3d4","Type":"ContainerStarted","Data":"914ec6c9d785f2a48d3b8904406cd08ea774d5db2c5bb2986d8b8bd5c16a3357"} Jul 16 21:51:09.992428 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:09.992399 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-prwdk" event={"ID":"81e44120-3c18-4e07-ba65-7ce68c3255e8","Type":"ContainerStarted","Data":"f448543b228f97cf74df9b0aef24430f57cbf7c01e156ab534a36c7526de407d"} Jul 16 21:51:10.012711 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:10.012665 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-97n2z" podStartSLOduration=1.012651351 podStartE2EDuration="1.012651351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:10.011458074 +0000 UTC m=+265.344537133" watchObservedRunningTime="2026-07-16 21:51:10.012651351 +0000 UTC m=+265.345730359" Jul 16 21:51:10.770257 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:10.769505 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:10.770257 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:10.769564 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:10.770257 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:10.769752 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:51:10.770257 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:10.769822 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls podName:7c827cc4-1d7f-4955-a2e0-270c3f522dab nodeName:}" failed. No retries permitted until 2026-07-16 21:51:12.769792583 +0000 UTC m=+268.102871598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-bs7nw" (UID: "7c827cc4-1d7f-4955-a2e0-270c3f522dab") : secret "samples-operator-tls" not found Jul 16 21:51:10.770257 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:10.769886 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:10.770257 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:10.769917 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls podName:25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa nodeName:}" failed. No retries permitted until 2026-07-16 21:51:12.769907729 +0000 UTC m=+268.102986742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-64kl9" (UID: "25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:10.871022 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:10.870764 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:10.871022 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:10.870820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:10.871022 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:10.870936 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:51:10.871022 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:10.870977 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:12.870958247 +0000 UTC m=+268.204037261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : configmap references non-existent config key: service-ca.crt Jul 16 21:51:10.871022 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:10.870998 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:12.870988864 +0000 UTC m=+268.204067875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : secret "router-metrics-certs-default" not found Jul 16 21:51:12.787448 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:12.787361 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:12.787857 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:12.787480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:12.787857 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:12.787504 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:51:12.787857 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:12.787565 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls podName:7c827cc4-1d7f-4955-a2e0-270c3f522dab nodeName:}" failed. No retries permitted until 2026-07-16 21:51:16.787549921 +0000 UTC m=+272.120628933 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-bs7nw" (UID: "7c827cc4-1d7f-4955-a2e0-270c3f522dab") : secret "samples-operator-tls" not found Jul 16 21:51:12.787857 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:12.787570 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:12.787857 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:12.787625 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls podName:25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa nodeName:}" failed. No retries permitted until 2026-07-16 21:51:16.787597482 +0000 UTC m=+272.120676507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-64kl9" (UID: "25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:12.888691 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:12.888657 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:12.888860 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:12.888714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:12.888860 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:12.888813 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:51:12.888860 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:12.888831 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:16.888819087 +0000 UTC m=+272.221898095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : configmap references non-existent config key: service-ca.crt Jul 16 21:51:12.889025 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:12.888873 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:16.888855807 +0000 UTC m=+272.221934821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : secret "router-metrics-certs-default" not found Jul 16 21:51:13.006061 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.006027 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-prwdk" event={"ID":"81e44120-3c18-4e07-ba65-7ce68c3255e8","Type":"ContainerStarted","Data":"f472c6e97c4e7312d28c657cc9b389ff35c3094e6da6d3186a9757cfd4381408"} Jul 16 21:51:13.007536 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.007508 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" event={"ID":"b738ba0e-3834-4ff6-9f6b-15d5f79a132d","Type":"ContainerStarted","Data":"e2ef99fae8a709206906cdbfe45ea7ea4575b2fea879f03ea433b3b3b8c81066"} Jul 16 21:51:13.009092 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.009072 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/0.log" Jul 16 21:51:13.009225 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.009103 2563 generic.go:356] "Generic (PLEG): container finished" podID="d91210e8-ef77-43ac-8287-c51e70dddbd9" containerID="62f1acda33cef98f06c2fe6017436839d1059342e13cbeabe25798586ebbdf7b" exitCode=255 Jul 16 21:51:13.009225 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.009125 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" event={"ID":"d91210e8-ef77-43ac-8287-c51e70dddbd9","Type":"ContainerDied","Data":"62f1acda33cef98f06c2fe6017436839d1059342e13cbeabe25798586ebbdf7b"} Jul 16 21:51:13.009371 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.009354 2563 scope.go:117] "RemoveContainer" containerID="62f1acda33cef98f06c2fe6017436839d1059342e13cbeabe25798586ebbdf7b" Jul 16 21:51:13.029074 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.029029 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-prwdk" podStartSLOduration=1.063561748 podStartE2EDuration="4.029011633s" totalImagesPullingTime="2.965449885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:13.026778762 +0000 UTC m=+268.359857819" watchObservedRunningTime="2026-07-16 21:51:13.029011633 +0000 UTC m=+268.362090664" Jul 16 21:51:13.062343 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.062295 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" podStartSLOduration=1.164167578 podStartE2EDuration="4.062281215s" totalImagesPullingTime="2.898113637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:13.061795436 +0000 UTC m=+268.394874443" watchObservedRunningTime="2026-07-16 21:51:13.062281215 +0000 UTC m=+268.395360268" Jul 16 21:51:13.852842 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.852807 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n5c8c"] Jul 16 21:51:13.856402 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.856377 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:13.859610 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.859586 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 21:51:13.859720 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.859651 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jvlkz\"" Jul 16 21:51:13.860935 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.860912 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 21:51:13.872094 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.872072 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5c8c"] Jul 16 21:51:13.999180 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.999128 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-crio-socket\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:13.999334 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.999199 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-data-volume\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:13.999334 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.999257 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:13.999409 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.999332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mlj9\" (UniqueName: \"kubernetes.io/projected/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-api-access-7mlj9\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:13.999409 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:13.999377 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.013353 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.013329 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 21:51:14.013701 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.013687 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/0.log" Jul 16 21:51:14.013742 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.013719 2563 generic.go:356] "Generic (PLEG): container finished" podID="d91210e8-ef77-43ac-8287-c51e70dddbd9" containerID="f1cadc2536ef15d4aa01dc3c643840fab12f45064a1afa869498f660c83824a1" exitCode=255 Jul 16 21:51:14.013771 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.013751 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" event={"ID":"d91210e8-ef77-43ac-8287-c51e70dddbd9","Type":"ContainerDied","Data":"f1cadc2536ef15d4aa01dc3c643840fab12f45064a1afa869498f660c83824a1"} Jul 16 21:51:14.013818 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.013811 2563 scope.go:117] "RemoveContainer" containerID="62f1acda33cef98f06c2fe6017436839d1059342e13cbeabe25798586ebbdf7b" Jul 16 21:51:14.014032 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.014007 2563 scope.go:117] "RemoveContainer" containerID="f1cadc2536ef15d4aa01dc3c643840fab12f45064a1afa869498f660c83824a1" Jul 16 21:51:14.014213 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.014191 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-8vbpp_openshift-console-operator(d91210e8-ef77-43ac-8287-c51e70dddbd9)\"" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" podUID="d91210e8-ef77-43ac-8287-c51e70dddbd9" Jul 16 21:51:14.100316 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.100290 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mlj9\" (UniqueName: \"kubernetes.io/projected/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-api-access-7mlj9\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.100478 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.100332 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.100478 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.100365 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-crio-socket\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.100478 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.100405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-data-volume\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.100478 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.100430 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:51:14.100478 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.100482 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:14.600468223 +0000 UTC m=+269.933547231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : configmap "kube-rbac-proxy" not found Jul 16 21:51:14.100703 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.100520 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-crio-socket\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.100703 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.100557 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.100703 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.100622 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:51:14.100703 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.100657 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:14.600648195 +0000 UTC m=+269.933727204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : secret "insights-runtime-extractor-tls" not found Jul 16 21:51:14.100833 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.100746 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-data-volume\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.112235 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.112173 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mlj9\" (UniqueName: \"kubernetes.io/projected/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-api-access-7mlj9\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.607057 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.603827 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.607057 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:14.603996 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:14.607057 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.604269 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:51:14.607057 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.604345 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:15.604327136 +0000 UTC m=+270.937406144 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : secret "insights-runtime-extractor-tls" not found Jul 16 21:51:14.607057 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.604778 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:51:14.607057 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:14.604832 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:15.604818102 +0000 UTC m=+270.937897125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : configmap "kube-rbac-proxy" not found Jul 16 21:51:15.019530 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:15.019452 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 21:51:15.019864 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:15.019764 2563 scope.go:117] "RemoveContainer" containerID="f1cadc2536ef15d4aa01dc3c643840fab12f45064a1afa869498f660c83824a1" Jul 16 21:51:15.019926 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:15.019906 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-8vbpp_openshift-console-operator(d91210e8-ef77-43ac-8287-c51e70dddbd9)\"" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" podUID="d91210e8-ef77-43ac-8287-c51e70dddbd9" Jul 16 21:51:15.613579 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:15.613546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:15.613767 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:15.613625 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:51:15.613767 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:15.613699 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:17.613675031 +0000 UTC m=+272.946754052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : configmap "kube-rbac-proxy" not found Jul 16 21:51:15.613767 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:15.613636 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:15.613767 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:15.613699 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:51:15.613983 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:15.613772 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:17.613762133 +0000 UTC m=+272.946841141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : secret "insights-runtime-extractor-tls" not found Jul 16 21:51:16.645384 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:16.645358 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fpm78_c3dbc476-9c43-48c3-a2bf-78a67110c028/dns-node-resolver/0.log" Jul 16 21:51:16.823556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:16.823522 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:16.823714 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:16.823638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:16.823714 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:16.823679 2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 21:51:16.823782 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:16.823718 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:16.823782 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:16.823748 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls podName:7c827cc4-1d7f-4955-a2e0-270c3f522dab nodeName:}" failed. No retries permitted until 2026-07-16 21:51:24.8237319 +0000 UTC m=+280.156810929 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-bs7nw" (UID: "7c827cc4-1d7f-4955-a2e0-270c3f522dab") : secret "samples-operator-tls" not found Jul 16 21:51:16.823782 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:16.823763 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls podName:25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa nodeName:}" failed. No retries permitted until 2026-07-16 21:51:24.82375702 +0000 UTC m=+280.156836029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-64kl9" (UID: "25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:16.924042 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:16.923976 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:16.924042 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:16.924038 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:16.924217 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:16.924124 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:51:16.924217 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:16.924200 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:24.924187498 +0000 UTC m=+280.257266506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : configmap references non-existent config key: service-ca.crt Jul 16 21:51:16.924217 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:16.924216 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:24.924210506 +0000 UTC m=+280.257289513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : secret "router-metrics-certs-default" not found Jul 16 21:51:17.627251 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:17.627203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:17.627455 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:17.627317 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:51:17.627455 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:17.627341 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:17.627455 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:17.627449 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:21.627431482 +0000 UTC m=+276.960510491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : configmap "kube-rbac-proxy" not found Jul 16 21:51:17.627578 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:17.627479 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:51:17.627578 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:17.627534 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:21.627518737 +0000 UTC m=+276.960597763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : secret "insights-runtime-extractor-tls" not found Jul 16 21:51:17.653582 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:17.653560 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pzmdz_a160cbfd-0e76-4746-9d1a-e442b9dbdff1/node-ca/0.log" Jul 16 21:51:18.645307 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:18.645273 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f22kb_d366d608-8179-4e70-8529-6dce72b0b665/serve-healthcheck-canary/0.log" Jul 16 21:51:19.358772 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:19.358748 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:19.359116 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:19.358797 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:19.359116 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:19.359077 2563 scope.go:117] "RemoveContainer" containerID="f1cadc2536ef15d4aa01dc3c643840fab12f45064a1afa869498f660c83824a1" Jul 16 21:51:19.359250 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:19.359231 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-8vbpp_openshift-console-operator(d91210e8-ef77-43ac-8287-c51e70dddbd9)\"" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" podUID="d91210e8-ef77-43ac-8287-c51e70dddbd9" Jul 16 21:51:21.658214 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:21.658177 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:21.658597 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:21.658256 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:21.658597 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:21.658316 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:51:21.658597 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:21.658337 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 21:51:21.658597 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:21.658377 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:29.658362738 +0000 UTC m=+284.991441746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : secret "insights-runtime-extractor-tls" not found Jul 16 21:51:21.658597 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:21.658391 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:29.658384994 +0000 UTC m=+284.991464002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : configmap "kube-rbac-proxy" not found Jul 16 21:51:24.884970 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:24.884939 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:24.885447 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:24.884979 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:24.885447 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:24.885116 2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:24.885447 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:24.885217 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls podName:25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa nodeName:}" failed. No retries permitted until 2026-07-16 21:51:40.885195579 +0000 UTC m=+296.218274591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-64kl9" (UID: "25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa") : secret "cluster-monitoring-operator-tls" not found Jul 16 21:51:24.887299 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:24.887281 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7c827cc4-1d7f-4955-a2e0-270c3f522dab-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-bs7nw\" (UID: \"7c827cc4-1d7f-4955-a2e0-270c3f522dab\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:24.945834 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:24.945807 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" Jul 16 21:51:24.986123 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:24.986090 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:24.986277 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:24.986261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:24.986368 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:24.986357 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:40.986337154 +0000 UTC m=+296.319416168 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : configmap references non-existent config key: service-ca.crt Jul 16 21:51:24.986414 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:24.986401 2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 21:51:24.986462 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:24.986456 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs podName:838fa3a2-b821-45b7-9678-a60fa9e797f2 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:40.986440044 +0000 UTC m=+296.319519061 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs") pod "router-default-544c55cfbb-t62g9" (UID: "838fa3a2-b821-45b7-9678-a60fa9e797f2") : secret "router-metrics-certs-default" not found Jul 16 21:51:25.098399 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:25.098367 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw"] Jul 16 21:51:26.051874 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:26.051828 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" event={"ID":"7c827cc4-1d7f-4955-a2e0-270c3f522dab","Type":"ContainerStarted","Data":"3498ae436e8ebecbd34bb7826b8459309eb71b04e7c197b1ed0a7c7e43e30900"} Jul 16 21:51:26.740136 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:26.740095 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-58f7c6584-lw844" podUID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" Jul 16 21:51:26.740289 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:26.740102 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vms7h" podUID="963b9f29-793e-4f12-a7bc-426a7b06dd53" Jul 16 21:51:27.056838 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:27.056800 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" event={"ID":"7c827cc4-1d7f-4955-a2e0-270c3f522dab","Type":"ContainerStarted","Data":"0c8da1eb2480e625be91714f1b1e685874ae6a879794b523c98db44a24512a14"} Jul 16 21:51:27.057310 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:27.056849 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:51:27.057310 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:27.056864 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" event={"ID":"7c827cc4-1d7f-4955-a2e0-270c3f522dab","Type":"ContainerStarted","Data":"c2647f8e6947ad46a952c9ad4b826c5a7a82bc18a571f8ba30a9a6f689c49311"} Jul 16 21:51:27.057310 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:27.056819 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vms7h" Jul 16 21:51:27.079201 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:27.079138 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-bs7nw" podStartSLOduration=16.725010076 podStartE2EDuration="18.079126329s" totalImagesPullingTime="1.354116253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:27.078646037 +0000 UTC m=+282.411725091" watchObservedRunningTime="2026-07-16 21:51:27.079126329 +0000 UTC m=+282.412205399" Jul 16 21:51:29.724308 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:29.724279 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:29.724754 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:29.724366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:29.724754 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:29.724480 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 21:51:29.724754 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:29.724543 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls podName:bad1b8fe-5a74-4b1c-a740-06a9c49128c0 nodeName:}" failed. No retries permitted until 2026-07-16 21:51:45.724526726 +0000 UTC m=+301.057605735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5c8c" (UID: "bad1b8fe-5a74-4b1c-a740-06a9c49128c0") : secret "insights-runtime-extractor-tls" not found Jul 16 21:51:29.724941 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:29.724922 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:30.429301 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.429246 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:51:30.429507 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.429366 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:51:30.431620 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.431595 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/963b9f29-793e-4f12-a7bc-426a7b06dd53-metrics-tls\") pod \"dns-default-vms7h\" (UID: \"963b9f29-793e-4f12-a7bc-426a7b06dd53\") " pod="openshift-dns/dns-default-vms7h" Jul 16 21:51:30.431804 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.431784 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"image-registry-58f7c6584-lw844\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:51:30.663093 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.663068 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-275zl\"" Jul 16 21:51:30.663093 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.663069 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j8r94\"" Jul 16 21:51:30.668110 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.668094 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:51:30.668211 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.668197 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vms7h" Jul 16 21:51:30.811805 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.811779 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vms7h"] Jul 16 21:51:30.815740 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:30.815715 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963b9f29_793e_4f12_a7bc_426a7b06dd53.slice/crio-f3eaad7bed996640d0875d161504696968c9134325623492a1b75f1a26ef5d92 WatchSource:0}: Error finding container f3eaad7bed996640d0875d161504696968c9134325623492a1b75f1a26ef5d92: Status 404 returned error can't find the container with id f3eaad7bed996640d0875d161504696968c9134325623492a1b75f1a26ef5d92 Jul 16 21:51:30.839632 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:30.839613 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58f7c6584-lw844"] Jul 16 21:51:30.841288 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:30.841258 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f0e0b2_de7f_46f9_b428_f0b1e1546653.slice/crio-442bee796dbe600f24af366b682a17ea01cfd898fd4c58061d4ef9768684a6d8 WatchSource:0}: Error finding container 442bee796dbe600f24af366b682a17ea01cfd898fd4c58061d4ef9768684a6d8: Status 404 returned error can't find the container with id 442bee796dbe600f24af366b682a17ea01cfd898fd4c58061d4ef9768684a6d8 Jul 16 21:51:31.069719 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:31.069642 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vms7h" event={"ID":"963b9f29-793e-4f12-a7bc-426a7b06dd53","Type":"ContainerStarted","Data":"f3eaad7bed996640d0875d161504696968c9134325623492a1b75f1a26ef5d92"} Jul 16 21:51:31.071062 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:31.071036 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58f7c6584-lw844" event={"ID":"91f0e0b2-de7f-46f9-b428-f0b1e1546653","Type":"ContainerStarted","Data":"7179493e96c5454fba3300b5668fa9113233c0be729e43740f8b8469715e9e9c"} Jul 16 21:51:31.071197 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:31.071074 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58f7c6584-lw844" event={"ID":"91f0e0b2-de7f-46f9-b428-f0b1e1546653","Type":"ContainerStarted","Data":"442bee796dbe600f24af366b682a17ea01cfd898fd4c58061d4ef9768684a6d8"} Jul 16 21:51:31.071197 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:31.071182 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:51:31.112716 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:31.112662 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58f7c6584-lw844" podStartSLOduration=285.112646771 podStartE2EDuration="4m45.112646771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:31.11253164 +0000 UTC m=+286.445610661" watchObservedRunningTime="2026-07-16 21:51:31.112646771 +0000 UTC m=+286.445725792" Jul 16 21:51:33.079277 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:33.079239 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vms7h" event={"ID":"963b9f29-793e-4f12-a7bc-426a7b06dd53","Type":"ContainerStarted","Data":"3bd63f4b6e9ab369b79cd0f679cdc3e869633edfc4153f5619a2e177aa20b4ff"} Jul 16 21:51:33.079658 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:33.079304 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vms7h" event={"ID":"963b9f29-793e-4f12-a7bc-426a7b06dd53","Type":"ContainerStarted","Data":"db40263d58a52c8763eb1f89e4520f21ace06839b04a14e8ffc3c4044bb738da"} Jul 16 21:51:33.079658 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:33.079372 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vms7h" Jul 16 21:51:33.102958 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:33.102915 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vms7h" podStartSLOduration=251.431914176 podStartE2EDuration="4m13.102902436s" totalImagesPullingTime="1.67098826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:33.102411059 +0000 UTC m=+288.435490080" watchObservedRunningTime="2026-07-16 21:51:33.102902436 +0000 UTC m=+288.435981466" Jul 16 21:51:34.235250 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:34.235217 2563 scope.go:117] "RemoveContainer" containerID="f1cadc2536ef15d4aa01dc3c643840fab12f45064a1afa869498f660c83824a1" Jul 16 21:51:35.087444 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:35.087415 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 21:51:35.087721 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:35.087467 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" event={"ID":"d91210e8-ef77-43ac-8287-c51e70dddbd9","Type":"ContainerStarted","Data":"d7857ef46443cb0eef5868f866607f156b79f78c3d3a041fe294a39b786d025b"} Jul 16 21:51:35.087721 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:35.087709 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:35.092352 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:35.092328 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" Jul 16 21:51:35.144127 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:35.144084 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-8vbpp" podStartSLOduration=23.167749484 podStartE2EDuration="26.144071288s" totalImagesPullingTime="2.976321804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:35.144053898 +0000 UTC m=+290.477132953" watchObservedRunningTime="2026-07-16 21:51:35.144071288 +0000 UTC m=+290.477150317" Jul 16 21:51:38.236054 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:38.236017 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xm229" podUID="ef310c53-4c7a-4331-91b9-6e7a2bb27841" Jul 16 21:51:38.836790 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:38.836756 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-j8k94"] Jul 16 21:51:38.839665 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:38.839649 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-j8k94" Jul 16 21:51:38.844781 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:38.844759 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 21:51:38.845036 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:38.845018 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cxnqv\"" Jul 16 21:51:38.845783 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:38.845613 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 21:51:38.867573 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:38.867543 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-j8k94"] Jul 16 21:51:38.900771 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:38.900748 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lntsn\" (UniqueName: \"kubernetes.io/projected/56a8954a-62c7-45c7-92dd-fe4421a8fe9f-kube-api-access-lntsn\") pod \"downloads-5bc67dcd46-j8k94\" (UID: \"56a8954a-62c7-45c7-92dd-fe4421a8fe9f\") " pod="openshift-console/downloads-5bc67dcd46-j8k94" Jul 16 21:51:39.001418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:39.001386 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lntsn\" (UniqueName: \"kubernetes.io/projected/56a8954a-62c7-45c7-92dd-fe4421a8fe9f-kube-api-access-lntsn\") pod \"downloads-5bc67dcd46-j8k94\" (UID: \"56a8954a-62c7-45c7-92dd-fe4421a8fe9f\") " pod="openshift-console/downloads-5bc67dcd46-j8k94" Jul 16 21:51:39.022029 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:39.022001 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lntsn\" (UniqueName: \"kubernetes.io/projected/56a8954a-62c7-45c7-92dd-fe4421a8fe9f-kube-api-access-lntsn\") pod \"downloads-5bc67dcd46-j8k94\" (UID: \"56a8954a-62c7-45c7-92dd-fe4421a8fe9f\") " pod="openshift-console/downloads-5bc67dcd46-j8k94" Jul 16 21:51:39.148517 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:39.148486 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-j8k94" Jul 16 21:51:39.293515 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:39.293443 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-j8k94"] Jul 16 21:51:39.295878 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:39.295855 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a8954a_62c7_45c7_92dd_fe4421a8fe9f.slice/crio-3e141cab5474f7a0f41f072a1b6a1322746702e7a189b3840beaa85df9518503 WatchSource:0}: Error finding container 3e141cab5474f7a0f41f072a1b6a1322746702e7a189b3840beaa85df9518503: Status 404 returned error can't find the container with id 3e141cab5474f7a0f41f072a1b6a1322746702e7a189b3840beaa85df9518503 Jul 16 21:51:40.108155 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:40.108114 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-j8k94" event={"ID":"56a8954a-62c7-45c7-92dd-fe4421a8fe9f","Type":"ContainerStarted","Data":"3e141cab5474f7a0f41f072a1b6a1322746702e7a189b3840beaa85df9518503"} Jul 16 21:51:40.918278 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:40.918239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:40.920932 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:40.920907 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-64kl9\" (UID: \"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:41.019489 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:41.019451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:41.019679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:41.019566 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:41.020157 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:41.020119 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/838fa3a2-b821-45b7-9678-a60fa9e797f2-service-ca-bundle\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:41.022162 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:41.022118 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/838fa3a2-b821-45b7-9678-a60fa9e797f2-metrics-certs\") pod \"router-default-544c55cfbb-t62g9\" (UID: \"838fa3a2-b821-45b7-9678-a60fa9e797f2\") " pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:41.167210 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:41.167167 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" Jul 16 21:51:41.249466 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:41.249437 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:41.324135 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:41.324108 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9"] Jul 16 21:51:41.327668 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:41.327641 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25674ffd_9e77_4e5a_aa0f_e2d8ab09c4aa.slice/crio-8b819c5df62d82c189e6b33c98693c3a881e24b0398e7aff0af798c38c12eb01 WatchSource:0}: Error finding container 8b819c5df62d82c189e6b33c98693c3a881e24b0398e7aff0af798c38c12eb01: Status 404 returned error can't find the container with id 8b819c5df62d82c189e6b33c98693c3a881e24b0398e7aff0af798c38c12eb01 Jul 16 21:51:41.405560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:41.405532 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-544c55cfbb-t62g9"] Jul 16 21:51:41.408324 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:41.408293 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838fa3a2_b821_45b7_9678_a60fa9e797f2.slice/crio-72ce0432e7f3265cf8bcbb64c75748965af4fd7611d0685cc39cfed88050251a WatchSource:0}: Error finding container 72ce0432e7f3265cf8bcbb64c75748965af4fd7611d0685cc39cfed88050251a: Status 404 returned error can't find the container with id 72ce0432e7f3265cf8bcbb64c75748965af4fd7611d0685cc39cfed88050251a Jul 16 21:51:42.118205 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:42.118169 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-544c55cfbb-t62g9" event={"ID":"838fa3a2-b821-45b7-9678-a60fa9e797f2","Type":"ContainerStarted","Data":"486689408e34c88c34a028270e058e3f9fcf891a7fbe49d3473aca5d7b595489"} Jul 16 21:51:42.118679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:42.118227 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-544c55cfbb-t62g9" event={"ID":"838fa3a2-b821-45b7-9678-a60fa9e797f2","Type":"ContainerStarted","Data":"72ce0432e7f3265cf8bcbb64c75748965af4fd7611d0685cc39cfed88050251a"} Jul 16 21:51:42.119383 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:42.119353 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" event={"ID":"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa","Type":"ContainerStarted","Data":"8b819c5df62d82c189e6b33c98693c3a881e24b0398e7aff0af798c38c12eb01"} Jul 16 21:51:42.145132 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:42.145086 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-544c55cfbb-t62g9" podStartSLOduration=33.145071468 podStartE2EDuration="33.145071468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:42.144540318 +0000 UTC m=+297.477619349" watchObservedRunningTime="2026-07-16 21:51:42.145071468 +0000 UTC m=+297.478150498" Jul 16 21:51:42.250157 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:42.250102 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:42.252928 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:42.252905 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:43.085513 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.085483 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vms7h" Jul 16 21:51:43.109471 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.107069 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bb598b595-bcdtr"] Jul 16 21:51:43.112446 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.112394 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.116223 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.115969 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 21:51:43.116223 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.115999 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 21:51:43.116223 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.116100 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 21:51:43.116464 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.116242 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 21:51:43.116464 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.116326 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 21:51:43.116558 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.116527 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dbx5n\"" Jul 16 21:51:43.125699 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.125638 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:43.126102 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.126069 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb598b595-bcdtr"] Jul 16 21:51:43.129697 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.129644 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-544c55cfbb-t62g9" Jul 16 21:51:43.242187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.239418 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-service-ca\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.242187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.239535 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vdgn\" (UniqueName: \"kubernetes.io/projected/3be744f6-6352-476f-be6b-825aa2e559ff-kube-api-access-9vdgn\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.242187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.239594 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-serving-cert\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.242187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.239641 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-oauth-config\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.242187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.239665 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-console-config\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.242187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.239688 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-oauth-serving-cert\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.341213 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.340365 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-service-ca\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.341213 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.340445 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vdgn\" (UniqueName: \"kubernetes.io/projected/3be744f6-6352-476f-be6b-825aa2e559ff-kube-api-access-9vdgn\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.341213 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.340485 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-serving-cert\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.341213 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.340528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-oauth-config\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.341213 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.340552 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-console-config\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.341213 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.340577 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-oauth-serving-cert\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.342138 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.341893 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-oauth-serving-cert\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.343595 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.343415 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-console-config\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.344069 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.344030 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-service-ca\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.346308 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.346267 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-oauth-config\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.346408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.346395 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-serving-cert\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.357614 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.357561 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vdgn\" (UniqueName: \"kubernetes.io/projected/3be744f6-6352-476f-be6b-825aa2e559ff-kube-api-access-9vdgn\") pod \"console-bb598b595-bcdtr\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.430534 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.430499 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:43.819130 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:43.819011 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb598b595-bcdtr"] Jul 16 21:51:43.821902 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:43.821871 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3be744f6_6352_476f_be6b_825aa2e559ff.slice/crio-bd74338ac1308a449cf19e1655d3b2d9d765d7f1c2a9f4a63a70482aa53c3a4a WatchSource:0}: Error finding container bd74338ac1308a449cf19e1655d3b2d9d765d7f1c2a9f4a63a70482aa53c3a4a: Status 404 returned error can't find the container with id bd74338ac1308a449cf19e1655d3b2d9d765d7f1c2a9f4a63a70482aa53c3a4a Jul 16 21:51:44.130330 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.130266 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb598b595-bcdtr" event={"ID":"3be744f6-6352-476f-be6b-825aa2e559ff","Type":"ContainerStarted","Data":"bd74338ac1308a449cf19e1655d3b2d9d765d7f1c2a9f4a63a70482aa53c3a4a"} Jul 16 21:51:44.132342 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.132291 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" event={"ID":"25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa","Type":"ContainerStarted","Data":"b68816028423295106b1ad58cdaf75ee90ca36427469d461cda958ad0af9b3ae"} Jul 16 21:51:44.166838 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.166787 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-64kl9" podStartSLOduration=32.771987459 podStartE2EDuration="35.166770875s" totalImagesPullingTime="2.394783416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:44.166392418 +0000 UTC m=+299.499471468" watchObservedRunningTime="2026-07-16 21:51:44.166770875 +0000 UTC m=+299.499849905" Jul 16 21:51:44.373265 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.373234 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f"] Jul 16 21:51:44.376795 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.376772 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" Jul 16 21:51:44.381823 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.381763 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 16 21:51:44.382210 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.382181 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7nl5h\"" Jul 16 21:51:44.398770 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.398717 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f"] Jul 16 21:51:44.449113 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.449035 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b26f4b90-79eb-4401-884c-570fc4e149c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-96r9f\" (UID: \"b26f4b90-79eb-4401-884c-570fc4e149c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" Jul 16 21:51:44.550270 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.550233 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b26f4b90-79eb-4401-884c-570fc4e149c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-96r9f\" (UID: \"b26f4b90-79eb-4401-884c-570fc4e149c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" Jul 16 21:51:44.553200 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.553138 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b26f4b90-79eb-4401-884c-570fc4e149c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-96r9f\" (UID: \"b26f4b90-79eb-4401-884c-570fc4e149c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" Jul 16 21:51:44.689466 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.689386 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" Jul 16 21:51:44.847632 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:44.847600 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f"] Jul 16 21:51:44.850534 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:44.850502 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26f4b90_79eb_4401_884c_570fc4e149c7.slice/crio-7de4ebe8db8898d136c0282c67c58d590941395d6827a2a3c117e6aef78c7892 WatchSource:0}: Error finding container 7de4ebe8db8898d136c0282c67c58d590941395d6827a2a3c117e6aef78c7892: Status 404 returned error can't find the container with id 7de4ebe8db8898d136c0282c67c58d590941395d6827a2a3c117e6aef78c7892 Jul 16 21:51:45.031620 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.031540 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 21:51:45.032050 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.032025 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 21:51:45.038978 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.038957 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 21:51:45.040658 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.040632 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 21:51:45.050109 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.050083 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 21:51:45.137641 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.137542 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" event={"ID":"b26f4b90-79eb-4401-884c-570fc4e149c7","Type":"ContainerStarted","Data":"7de4ebe8db8898d136c0282c67c58d590941395d6827a2a3c117e6aef78c7892"} Jul 16 21:51:45.760207 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.760160 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:45.763296 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.763267 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad1b8fe-5a74-4b1c-a740-06a9c49128c0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5c8c\" (UID: \"bad1b8fe-5a74-4b1c-a740-06a9c49128c0\") " pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:45.971002 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.970958 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jvlkz\"" Jul 16 21:51:45.977133 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:45.977097 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5c8c" Jul 16 21:51:47.330308 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:47.330277 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5c8c"] Jul 16 21:51:47.333543 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:47.333522 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 21:51:48.151929 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:48.151891 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb598b595-bcdtr" event={"ID":"3be744f6-6352-476f-be6b-825aa2e559ff","Type":"ContainerStarted","Data":"5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc"} Jul 16 21:51:48.153878 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:48.153836 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5c8c" event={"ID":"bad1b8fe-5a74-4b1c-a740-06a9c49128c0","Type":"ContainerStarted","Data":"351df152584f67f66584f3b4bf738b5fe922c111123b88807f0fd87ba51f77b7"} Jul 16 21:51:48.155896 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:48.155868 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" event={"ID":"b26f4b90-79eb-4401-884c-570fc4e149c7","Type":"ContainerStarted","Data":"d7f92c5183e0747c3feec20fcdfb2b2840950468e5cb35ed9e482f7ad3be250c"} Jul 16 21:51:48.156392 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:48.156374 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" Jul 16 21:51:48.163922 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:48.163900 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" Jul 16 21:51:48.178358 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:48.177205 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bb598b595-bcdtr" podStartSLOduration=1.8287569860000001 podStartE2EDuration="5.177187157s" totalImagesPullingTime="3.348430171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:48.176538494 +0000 UTC m=+303.509617523" watchObservedRunningTime="2026-07-16 21:51:48.177187157 +0000 UTC m=+303.510266190" Jul 16 21:51:50.673899 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:50.673840 2563 patch_prober.go:28] interesting pod/image-registry-58f7c6584-lw844 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 21:51:50.674551 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:50.673912 2563 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-58f7c6584-lw844" podUID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 21:51:52.079834 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:52.079804 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:51:52.122745 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:52.122685 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-96r9f" podStartSLOduration=5.8036535879999995 podStartE2EDuration="8.122666359s" totalImagesPullingTime="2.319012771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:51:48.199862971 +0000 UTC m=+303.532942026" watchObservedRunningTime="2026-07-16 21:51:52.122666359 +0000 UTC m=+307.455745390" Jul 16 21:51:53.234377 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:53.234345 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:51:53.431210 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:53.431173 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:53.431394 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:53.431269 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:53.436580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:53.436555 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:54.183434 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.183401 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:51:54.879969 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.879938 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp"] Jul 16 21:51:54.883960 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.883932 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:54.888525 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.888490 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 16 21:51:54.888664 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.888527 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 16 21:51:54.888664 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.888529 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 21:51:54.888664 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.888528 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8q5fc\"" Jul 16 21:51:54.906168 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.905930 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp"] Jul 16 21:51:54.908378 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.908352 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rv9bc"] Jul 16 21:51:54.912821 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.912793 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:54.915873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.915852 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 21:51:54.915990 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.915869 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 21:51:54.915990 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.915887 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 21:51:54.916115 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:54.916002 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l4dh8\"" Jul 16 21:51:55.048049 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048007 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e647a04-81bc-44fc-b2f8-084efe58cee9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.048247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048079 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnxj5\" (UniqueName: \"kubernetes.io/projected/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-kube-api-access-mnxj5\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.048247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048116 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e647a04-81bc-44fc-b2f8-084efe58cee9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.048247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048153 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e647a04-81bc-44fc-b2f8-084efe58cee9-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.048247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048178 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-root\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.048247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048234 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-sys\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.048247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048249 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-wtmp\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.048586 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048274 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqp5d\" (UniqueName: \"kubernetes.io/projected/8e647a04-81bc-44fc-b2f8-084efe58cee9-kube-api-access-gqp5d\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.048586 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048301 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-tls\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.048586 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-textfile\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.048586 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048382 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.048586 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048416 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-metrics-client-ca\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.048586 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.048441 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.148873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.148787 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-textfile\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.148873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.148835 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.148873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.148866 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-metrics-client-ca\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.148881 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.148922 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e647a04-81bc-44fc-b2f8-084efe58cee9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.149112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.148968 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnxj5\" (UniqueName: \"kubernetes.io/projected/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-kube-api-access-mnxj5\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149004 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e647a04-81bc-44fc-b2f8-084efe58cee9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.149112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e647a04-81bc-44fc-b2f8-084efe58cee9-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.149112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149052 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-root\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149089 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-sys\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149110 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-wtmp\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-textfile\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149246 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-sys\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149279 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-wtmp\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149302 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-root\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqp5d\" (UniqueName: \"kubernetes.io/projected/8e647a04-81bc-44fc-b2f8-084efe58cee9-kube-api-access-gqp5d\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.149556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149343 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-tls\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.149556 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:55.149455 2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 16 21:51:55.149922 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:55.149775 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-tls podName:a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a nodeName:}" failed. No retries permitted until 2026-07-16 21:51:55.649745232 +0000 UTC m=+310.982824252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-tls") pod "node-exporter-rv9bc" (UID: "a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a") : secret "node-exporter-tls" not found Jul 16 21:51:55.149922 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.149854 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.150208 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.150185 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e647a04-81bc-44fc-b2f8-084efe58cee9-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.150364 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.150249 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-metrics-client-ca\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.152047 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.152020 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.152160 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.152111 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8e647a04-81bc-44fc-b2f8-084efe58cee9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.152515 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.152493 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e647a04-81bc-44fc-b2f8-084efe58cee9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.158339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.158295 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqp5d\" (UniqueName: \"kubernetes.io/projected/8e647a04-81bc-44fc-b2f8-084efe58cee9-kube-api-access-gqp5d\") pod \"openshift-state-metrics-79f6958cdf-8t2qp\" (UID: \"8e647a04-81bc-44fc-b2f8-084efe58cee9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.158661 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.158640 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnxj5\" (UniqueName: \"kubernetes.io/projected/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-kube-api-access-mnxj5\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.195179 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.195132 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" Jul 16 21:51:55.654505 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.654469 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-tls\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.657228 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.657192 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a-node-exporter-tls\") pod \"node-exporter-rv9bc\" (UID: \"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a\") " pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.824659 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.824633 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rv9bc" Jul 16 21:51:55.944704 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.944624 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 21:51:55.948814 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.948788 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:55.951969 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.951942 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 16 21:51:55.952092 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.951955 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 16 21:51:55.952092 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.952036 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 16 21:51:55.952230 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.951986 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 16 21:51:55.952230 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.952176 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 16 21:51:55.952500 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.952450 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 16 21:51:55.952500 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.952487 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 16 21:51:55.952634 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.952502 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 16 21:51:55.952634 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.952518 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-t9b4b\"" Jul 16 21:51:55.952770 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.952749 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 16 21:51:55.965365 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:55.965284 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 21:51:56.058155 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058120 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058340 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058179 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058340 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058233 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a566030-98b2-449f-aaed-1e6caaf3556a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058340 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058272 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-web-config\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058340 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058361 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a566030-98b2-449f-aaed-1e6caaf3556a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058440 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058462 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pp2d\" (UniqueName: \"kubernetes.io/projected/2a566030-98b2-449f-aaed-1e6caaf3556a-kube-api-access-2pp2d\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058493 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a566030-98b2-449f-aaed-1e6caaf3556a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058560 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058540 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a566030-98b2-449f-aaed-1e6caaf3556a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.058873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.058607 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a566030-98b2-449f-aaed-1e6caaf3556a-config-out\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.159936 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.159875 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a566030-98b2-449f-aaed-1e6caaf3556a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160103 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.159945 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a566030-98b2-449f-aaed-1e6caaf3556a-config-out\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160103 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.159989 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160103 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160022 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160103 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160047 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a566030-98b2-449f-aaed-1e6caaf3556a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160103 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160072 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-web-config\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160107 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160171 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160198 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a566030-98b2-449f-aaed-1e6caaf3556a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pp2d\" (UniqueName: \"kubernetes.io/projected/2a566030-98b2-449f-aaed-1e6caaf3556a-kube-api-access-2pp2d\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160318 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a566030-98b2-449f-aaed-1e6caaf3556a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160757 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160689 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2a566030-98b2-449f-aaed-1e6caaf3556a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.160809 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.160762 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a566030-98b2-449f-aaed-1e6caaf3556a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.162191 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.161665 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2a566030-98b2-449f-aaed-1e6caaf3556a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.165438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.164878 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.165438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.165069 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-web-config\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.165438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.165387 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.165664 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.165643 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2a566030-98b2-449f-aaed-1e6caaf3556a-config-out\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.165913 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.165885 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-config-volume\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.166245 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.166220 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.166721 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.166701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.166898 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.166873 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2a566030-98b2-449f-aaed-1e6caaf3556a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.166994 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.166982 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2a566030-98b2-449f-aaed-1e6caaf3556a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.179022 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.178996 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pp2d\" (UniqueName: \"kubernetes.io/projected/2a566030-98b2-449f-aaed-1e6caaf3556a-kube-api-access-2pp2d\") pod \"alertmanager-main-0\" (UID: \"2a566030-98b2-449f-aaed-1e6caaf3556a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.261302 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.261234 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 16 21:51:56.943453 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.943410 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7dcbc74488-9r7tl"] Jul 16 21:51:56.955982 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.955948 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:56.959813 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.959786 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 16 21:51:56.959813 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.959808 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-epul6042fj4ts\"" Jul 16 21:51:56.960006 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.959855 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 16 21:51:56.960006 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.959786 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 16 21:51:56.960196 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.960168 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zh449\"" Jul 16 21:51:56.960325 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.960264 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 16 21:51:56.960390 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.960380 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 16 21:51:56.960478 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:56.960456 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7dcbc74488-9r7tl"] Jul 16 21:51:57.069011 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.068978 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tggp6\" (UniqueName: \"kubernetes.io/projected/69971137-9cb9-42eb-8b18-5f4e828ff8c1-kube-api-access-tggp6\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.069262 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.069040 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-tls\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.069262 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.069070 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.069262 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.069097 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.069262 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.069129 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69971137-9cb9-42eb-8b18-5f4e828ff8c1-metrics-client-ca\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.069262 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.069171 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-grpc-tls\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.069464 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.069256 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.069464 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.069312 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.170573 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.170526 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tggp6\" (UniqueName: \"kubernetes.io/projected/69971137-9cb9-42eb-8b18-5f4e828ff8c1-kube-api-access-tggp6\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.170755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.170593 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-tls\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.170755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.170619 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.170755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.170645 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.170755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.170677 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69971137-9cb9-42eb-8b18-5f4e828ff8c1-metrics-client-ca\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.170755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.170703 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-grpc-tls\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.170755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.170748 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.171058 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.170795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.171959 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.171926 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69971137-9cb9-42eb-8b18-5f4e828ff8c1-metrics-client-ca\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.174075 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.174030 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-tls\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.174593 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.174564 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.174745 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.174717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-grpc-tls\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.174843 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.174717 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.174908 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.174883 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.175070 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.175048 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69971137-9cb9-42eb-8b18-5f4e828ff8c1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.179014 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.178995 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tggp6\" (UniqueName: \"kubernetes.io/projected/69971137-9cb9-42eb-8b18-5f4e828ff8c1-kube-api-access-tggp6\") pod \"thanos-querier-7dcbc74488-9r7tl\" (UID: \"69971137-9cb9-42eb-8b18-5f4e828ff8c1\") " pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:57.269003 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:57.268927 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:51:59.420302 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.420265 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d"] Jul 16 21:51:59.482338 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.482262 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d"] Jul 16 21:51:59.482473 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.482365 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.486534 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.486514 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 16 21:51:59.486819 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.486805 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 16 21:51:59.488271 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.488235 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-42i78bfjib50l\"" Jul 16 21:51:59.488402 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.488380 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jhhf7\"" Jul 16 21:51:59.488476 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.488441 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 16 21:51:59.488476 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.488462 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 16 21:51:59.593584 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.593513 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-metrics-server-audit-profiles\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.593745 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.593615 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-audit-log\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.593745 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.593651 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.593857 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.593746 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-secret-metrics-server-client-certs\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.593857 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.593820 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9bhl\" (UniqueName: \"kubernetes.io/projected/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-kube-api-access-j9bhl\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.593857 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.593852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-client-ca-bundle\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.593991 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.593893 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-secret-metrics-server-tls\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.667699 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.667665 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj"] Jul 16 21:51:59.683267 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.683229 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:51:59.683458 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.683438 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj"] Jul 16 21:51:59.687855 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.687834 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qf92g\"" Jul 16 21:51:59.688180 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.688159 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 16 21:51:59.696175 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.696122 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-audit-log\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.696408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.696348 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.696649 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.696576 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-secret-metrics-server-client-certs\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.696808 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.696626 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhl\" (UniqueName: \"kubernetes.io/projected/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-kube-api-access-j9bhl\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.697046 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.696941 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-client-ca-bundle\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.697243 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.697016 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-secret-metrics-server-tls\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.697488 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.697220 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-metrics-server-audit-profiles\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.698810 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.698705 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-metrics-server-audit-profiles\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.700082 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.699134 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-audit-log\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.700082 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.699737 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.711010 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.710685 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-secret-metrics-server-client-certs\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.715211 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.712239 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-client-ca-bundle\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.722190 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.722167 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-secret-metrics-server-tls\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.725314 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.725291 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9bhl\" (UniqueName: \"kubernetes.io/projected/7f77f1c8-c8d6-4253-8d41-3fd27ca756fa-kube-api-access-j9bhl\") pod \"metrics-server-c8c4fdcb9-fhg9d\" (UID: \"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa\") " pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.791937 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.791855 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:51:59.798181 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.798137 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hdpg\" (UniqueName: \"kubernetes.io/projected/62eab82f-66fd-44ce-8dc0-4b61937eaeee-kube-api-access-2hdpg\") pod \"monitoring-plugin-7b7b7bc8bf-l8pqj\" (UID: \"62eab82f-66fd-44ce-8dc0-4b61937eaeee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:51:59.798330 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.798300 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62eab82f-66fd-44ce-8dc0-4b61937eaeee-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-l8pqj\" (UID: \"62eab82f-66fd-44ce-8dc0-4b61937eaeee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:51:59.899677 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.899638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62eab82f-66fd-44ce-8dc0-4b61937eaeee-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-l8pqj\" (UID: \"62eab82f-66fd-44ce-8dc0-4b61937eaeee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:51:59.899871 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.899698 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hdpg\" (UniqueName: \"kubernetes.io/projected/62eab82f-66fd-44ce-8dc0-4b61937eaeee-kube-api-access-2hdpg\") pod \"monitoring-plugin-7b7b7bc8bf-l8pqj\" (UID: \"62eab82f-66fd-44ce-8dc0-4b61937eaeee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:51:59.899871 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:59.899822 2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 16 21:51:59.899989 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:51:59.899906 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62eab82f-66fd-44ce-8dc0-4b61937eaeee-monitoring-plugin-cert podName:62eab82f-66fd-44ce-8dc0-4b61937eaeee nodeName:}" failed. No retries permitted until 2026-07-16 21:52:00.399884594 +0000 UTC m=+315.732963602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/62eab82f-66fd-44ce-8dc0-4b61937eaeee-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-l8pqj" (UID: "62eab82f-66fd-44ce-8dc0-4b61937eaeee") : secret "monitoring-plugin-cert" not found Jul 16 21:51:59.912942 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:51:59.912917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hdpg\" (UniqueName: \"kubernetes.io/projected/62eab82f-66fd-44ce-8dc0-4b61937eaeee-kube-api-access-2hdpg\") pod \"monitoring-plugin-7b7b7bc8bf-l8pqj\" (UID: \"62eab82f-66fd-44ce-8dc0-4b61937eaeee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:51:59.993088 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:51:59.993048 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a3b611_c94b_4f8a_9ddb_a15ac7feb80a.slice/crio-894ffd26fe495234d6c87e6b64fbb15086dff4369d908249d9cb95f9d6646fa8 WatchSource:0}: Error finding container 894ffd26fe495234d6c87e6b64fbb15086dff4369d908249d9cb95f9d6646fa8: Status 404 returned error can't find the container with id 894ffd26fe495234d6c87e6b64fbb15086dff4369d908249d9cb95f9d6646fa8 Jul 16 21:52:00.162279 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.161968 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7dcbc74488-9r7tl"] Jul 16 21:52:00.164792 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:52:00.164761 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69971137_9cb9_42eb_8b18_5f4e828ff8c1.slice/crio-daeca8b15f54ce9daca26defac225153d392be27541b5bec1c91471cd225ddaf WatchSource:0}: Error finding container daeca8b15f54ce9daca26defac225153d392be27541b5bec1c91471cd225ddaf: Status 404 returned error can't find the container with id daeca8b15f54ce9daca26defac225153d392be27541b5bec1c91471cd225ddaf Jul 16 21:52:00.186423 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.186361 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp"] Jul 16 21:52:00.188383 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:52:00.188358 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e647a04_81bc_44fc_b2f8_084efe58cee9.slice/crio-63e835441d8be61886ea01d877fccb9646d96f5e40ae86daa293e0c0332639bf WatchSource:0}: Error finding container 63e835441d8be61886ea01d877fccb9646d96f5e40ae86daa293e0c0332639bf: Status 404 returned error can't find the container with id 63e835441d8be61886ea01d877fccb9646d96f5e40ae86daa293e0c0332639bf Jul 16 21:52:00.202904 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.202879 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" event={"ID":"69971137-9cb9-42eb-8b18-5f4e828ff8c1","Type":"ContainerStarted","Data":"daeca8b15f54ce9daca26defac225153d392be27541b5bec1c91471cd225ddaf"} Jul 16 21:52:00.204192 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.204169 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv9bc" event={"ID":"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a","Type":"ContainerStarted","Data":"894ffd26fe495234d6c87e6b64fbb15086dff4369d908249d9cb95f9d6646fa8"} Jul 16 21:52:00.205892 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.205867 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-j8k94" event={"ID":"56a8954a-62c7-45c7-92dd-fe4421a8fe9f","Type":"ContainerStarted","Data":"9fc5904e6ea0c11fc79fe6ba455b9eaff88edf1f72539a82bbe4a016d5b2d5a7"} Jul 16 21:52:00.206245 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.206069 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-j8k94" Jul 16 21:52:00.207112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.207090 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" event={"ID":"8e647a04-81bc-44fc-b2f8-084efe58cee9","Type":"ContainerStarted","Data":"63e835441d8be61886ea01d877fccb9646d96f5e40ae86daa293e0c0332639bf"} Jul 16 21:52:00.207651 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.207630 2563 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-j8k94 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.17:8080/\": dial tcp 10.132.0.17:8080: connect: connection refused" start-of-body= Jul 16 21:52:00.207746 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.207671 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-j8k94" podUID="56a8954a-62c7-45c7-92dd-fe4421a8fe9f" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.17:8080/\": dial tcp 10.132.0.17:8080: connect: connection refused" Jul 16 21:52:00.404101 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.404049 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62eab82f-66fd-44ce-8dc0-4b61937eaeee-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-l8pqj\" (UID: \"62eab82f-66fd-44ce-8dc0-4b61937eaeee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:52:00.406820 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.406535 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/62eab82f-66fd-44ce-8dc0-4b61937eaeee-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-l8pqj\" (UID: \"62eab82f-66fd-44ce-8dc0-4b61937eaeee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:52:00.417571 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.417388 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-j8k94" podStartSLOduration=1.736108462 podStartE2EDuration="22.417366846s" totalImagesPullingTime="20.681258384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:00.282442163 +0000 UTC m=+315.615521193" watchObservedRunningTime="2026-07-16 21:52:00.417366846 +0000 UTC m=+315.750445868" Jul 16 21:52:00.419615 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.419485 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d"] Jul 16 21:52:00.425126 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:52:00.425098 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f77f1c8_c8d6_4253_8d41_3fd27ca756fa.slice/crio-b4ecab5ae1000ad608caf93b5284bd182ed57700a23dfaddc5982acf46eb622d WatchSource:0}: Error finding container b4ecab5ae1000ad608caf93b5284bd182ed57700a23dfaddc5982acf46eb622d: Status 404 returned error can't find the container with id b4ecab5ae1000ad608caf93b5284bd182ed57700a23dfaddc5982acf46eb622d Jul 16 21:52:00.430810 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.430786 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 16 21:52:00.436344 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:52:00.436314 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a566030_98b2_449f_aaed_1e6caaf3556a.slice/crio-67d3054b3ef231069b5336dac8900bc3346381e801a714d9dec6aa6529fcddcf WatchSource:0}: Error finding container 67d3054b3ef231069b5336dac8900bc3346381e801a714d9dec6aa6529fcddcf: Status 404 returned error can't find the container with id 67d3054b3ef231069b5336dac8900bc3346381e801a714d9dec6aa6529fcddcf Jul 16 21:52:00.601177 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.601067 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:52:00.784272 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:00.784232 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj"] Jul 16 21:52:00.796833 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:52:00.796797 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62eab82f_66fd_44ce_8dc0_4b61937eaeee.slice/crio-5df086d83036ae0def42d14d80b43fdb94356086a6682b198f89ff656f3d3399 WatchSource:0}: Error finding container 5df086d83036ae0def42d14d80b43fdb94356086a6682b198f89ff656f3d3399: Status 404 returned error can't find the container with id 5df086d83036ae0def42d14d80b43fdb94356086a6682b198f89ff656f3d3399 Jul 16 21:52:01.171711 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:01.170936 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58f7c6584-lw844"] Jul 16 21:52:01.224251 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:01.224132 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5c8c" event={"ID":"bad1b8fe-5a74-4b1c-a740-06a9c49128c0","Type":"ContainerStarted","Data":"8adf486cc3fa7c9c5c1080e0f263f56d3c28a49ed52dac44d85c9e34d0d69aa6"} Jul 16 21:52:01.244638 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:01.244540 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" event={"ID":"8e647a04-81bc-44fc-b2f8-084efe58cee9","Type":"ContainerStarted","Data":"878c34fec4a9bd37a26d5826d619ed8293fb240e1b24a1dd4969233a04cf7051"} Jul 16 21:52:01.244823 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:01.244675 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" event={"ID":"8e647a04-81bc-44fc-b2f8-084efe58cee9","Type":"ContainerStarted","Data":"a8c1681a0064db804aca1c7bf1d91799b06b271dd1852261cb33752975cdf4b9"} Jul 16 21:52:01.244823 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:01.244707 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" event={"ID":"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa","Type":"ContainerStarted","Data":"b4ecab5ae1000ad608caf93b5284bd182ed57700a23dfaddc5982acf46eb622d"} Jul 16 21:52:01.244823 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:01.244734 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a566030-98b2-449f-aaed-1e6caaf3556a","Type":"ContainerStarted","Data":"67d3054b3ef231069b5336dac8900bc3346381e801a714d9dec6aa6529fcddcf"} Jul 16 21:52:01.247265 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:01.247058 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" event={"ID":"62eab82f-66fd-44ce-8dc0-4b61937eaeee","Type":"ContainerStarted","Data":"5df086d83036ae0def42d14d80b43fdb94356086a6682b198f89ff656f3d3399"} Jul 16 21:52:01.256434 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:01.256405 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-j8k94" Jul 16 21:52:02.714900 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.714861 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bfd64b99b-ntf8t"] Jul 16 21:52:02.744976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.743251 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bfd64b99b-ntf8t"] Jul 16 21:52:02.744976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.743386 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.782805 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.782774 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 21:52:02.833283 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.832844 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-oauth-serving-cert\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.833283 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.832913 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-oauth-config\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.833283 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.832944 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-service-ca\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.833283 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.832976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w4kd\" (UniqueName: \"kubernetes.io/projected/4c660ea6-ed99-41db-92c9-326bb5814f76-kube-api-access-9w4kd\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.833283 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.833002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-serving-cert\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.833283 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.833049 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-console-config\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.833283 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.833071 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-trusted-ca-bundle\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.933842 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.933802 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4kd\" (UniqueName: \"kubernetes.io/projected/4c660ea6-ed99-41db-92c9-326bb5814f76-kube-api-access-9w4kd\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.933842 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.933847 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-serving-cert\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.934124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.933887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-console-config\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.934124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.933909 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-trusted-ca-bundle\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.934124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.934012 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-oauth-serving-cert\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.934124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.934055 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-oauth-config\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.934124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.934078 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-service-ca\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.934753 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.934728 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-service-ca\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.936853 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.935991 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-console-config\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.936853 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.936039 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-trusted-ca-bundle\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.936853 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.936550 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-oauth-serving-cert\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.939369 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.939310 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-serving-cert\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.941069 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.940996 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-oauth-config\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:02.948587 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:02.948546 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w4kd\" (UniqueName: \"kubernetes.io/projected/4c660ea6-ed99-41db-92c9-326bb5814f76-kube-api-access-9w4kd\") pod \"console-6bfd64b99b-ntf8t\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:03.066352 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:03.066243 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:09.075429 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.075396 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bfd64b99b-ntf8t"] Jul 16 21:52:09.086542 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:52:09.086505 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c660ea6_ed99_41db_92c9_326bb5814f76.slice/crio-3f6c5dfdafe40465c3b7e2ee5db552f8d40f292b06b16543d0a69d375eaff972 WatchSource:0}: Error finding container 3f6c5dfdafe40465c3b7e2ee5db552f8d40f292b06b16543d0a69d375eaff972: Status 404 returned error can't find the container with id 3f6c5dfdafe40465c3b7e2ee5db552f8d40f292b06b16543d0a69d375eaff972 Jul 16 21:52:09.295828 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.295797 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" event={"ID":"69971137-9cb9-42eb-8b18-5f4e828ff8c1","Type":"ContainerStarted","Data":"00a1558db945eb57e8b909943bda6a7fb8f4f271b6adac8b800e00402f1bc226"} Jul 16 21:52:09.297337 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.297309 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" event={"ID":"62eab82f-66fd-44ce-8dc0-4b61937eaeee","Type":"ContainerStarted","Data":"0ebff254da4cdadedd89ff7ebf0493708bf946b6e7c1bd00468bf4531d45a451"} Jul 16 21:52:09.297612 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.297589 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:52:09.299032 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.299010 2563 generic.go:356] "Generic (PLEG): container finished" podID="a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a" containerID="57b1937898e33f89afde719c9c281413969a4a18b6b5bbe1f2b48ffbb5ac50a7" exitCode=0 Jul 16 21:52:09.299124 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.299098 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv9bc" event={"ID":"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a","Type":"ContainerDied","Data":"57b1937898e33f89afde719c9c281413969a4a18b6b5bbe1f2b48ffbb5ac50a7"} Jul 16 21:52:09.301192 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.301165 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bfd64b99b-ntf8t" event={"ID":"4c660ea6-ed99-41db-92c9-326bb5814f76","Type":"ContainerStarted","Data":"3f6c5dfdafe40465c3b7e2ee5db552f8d40f292b06b16543d0a69d375eaff972"} Jul 16 21:52:09.303605 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.303548 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" Jul 16 21:52:09.304962 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.304927 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" event={"ID":"8e647a04-81bc-44fc-b2f8-084efe58cee9","Type":"ContainerStarted","Data":"19f1a1c88f4158dace07cac9c90510441fd1c387fbf4ac0760a43faf9f18d719"} Jul 16 21:52:09.307630 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.307252 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" event={"ID":"7f77f1c8-c8d6-4253-8d41-3fd27ca756fa","Type":"ContainerStarted","Data":"731b85cf41b99148dc9f0e6b3db2ceec056306b603643be2e1c0a1af944cfbc7"} Jul 16 21:52:09.309077 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.309053 2563 generic.go:356] "Generic (PLEG): container finished" podID="2a566030-98b2-449f-aaed-1e6caaf3556a" containerID="184018c92c8df6727d35bdf37ab6e03e9bdeaa83ac9941542ac40127f754638e" exitCode=0 Jul 16 21:52:09.309195 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.309110 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a566030-98b2-449f-aaed-1e6caaf3556a","Type":"ContainerDied","Data":"184018c92c8df6727d35bdf37ab6e03e9bdeaa83ac9941542ac40127f754638e"} Jul 16 21:52:09.315574 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.315538 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-l8pqj" podStartSLOduration=2.194712232 podStartE2EDuration="10.315526677s" totalImagesPullingTime="8.120814445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:09.31394288 +0000 UTC m=+324.647021899" watchObservedRunningTime="2026-07-16 21:52:09.315526677 +0000 UTC m=+324.648605685" Jul 16 21:52:09.336796 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.335808 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" podStartSLOduration=1.842486786 podStartE2EDuration="10.335789195s" totalImagesPullingTime="8.493302409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:09.335766683 +0000 UTC m=+324.668845713" watchObservedRunningTime="2026-07-16 21:52:09.335789195 +0000 UTC m=+324.668868231" Jul 16 21:52:09.434769 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:09.434674 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t2qp" podStartSLOduration=6.991038038 podStartE2EDuration="15.434651314s" totalImagesPullingTime="8.443613276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:09.433738378 +0000 UTC m=+324.766817408" watchObservedRunningTime="2026-07-16 21:52:09.434651314 +0000 UTC m=+324.767730347" Jul 16 21:52:10.321572 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:10.321532 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bfd64b99b-ntf8t" event={"ID":"4c660ea6-ed99-41db-92c9-326bb5814f76","Type":"ContainerStarted","Data":"c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456"} Jul 16 21:52:10.324536 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:10.324503 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5c8c" event={"ID":"bad1b8fe-5a74-4b1c-a740-06a9c49128c0","Type":"ContainerStarted","Data":"9c2d591aff7bf08721f450ae682e7e671bb5577f808789ba550bed74c24f6ae8"} Jul 16 21:52:10.327383 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:10.327356 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" event={"ID":"69971137-9cb9-42eb-8b18-5f4e828ff8c1","Type":"ContainerStarted","Data":"c5a251f0285da4a9405c74cbb5c08c0eb451531cf3f8f729f647a5914380bb1b"} Jul 16 21:52:10.327501 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:10.327416 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" event={"ID":"69971137-9cb9-42eb-8b18-5f4e828ff8c1","Type":"ContainerStarted","Data":"0221685dac222ec96d3838286e1cdab4c9d3e586f020165dd8721faf25142437"} Jul 16 21:52:10.331344 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:10.331203 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv9bc" event={"ID":"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a","Type":"ContainerStarted","Data":"a63d1f97ac3a833fe88f7ce8293a58360918cbc9e5b33fbe3c7ac3d2b361ec7a"} Jul 16 21:52:10.331344 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:10.331262 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv9bc" event={"ID":"a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a","Type":"ContainerStarted","Data":"d5ff98646950c7b231f619d191656a0346f794324a69f0c3501efe510a73fc12"} Jul 16 21:52:10.361546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:10.361468 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bfd64b99b-ntf8t" podStartSLOduration=8.361449019 podStartE2EDuration="8.361449019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:52:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:10.358976207 +0000 UTC m=+325.692055262" watchObservedRunningTime="2026-07-16 21:52:10.361449019 +0000 UTC m=+325.694528052" Jul 16 21:52:10.394158 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:10.394089 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rv9bc" podStartSLOduration=8.525512238 podStartE2EDuration="16.394072597s" totalImagesPullingTime="7.768937073s" totalInitContainerRuntime="99.623286ms" isStatefulPod=true podCreationTimestamp="2026-07-16 21:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:10.392537662 +0000 UTC m=+325.725616688" watchObservedRunningTime="2026-07-16 21:52:10.394072597 +0000 UTC m=+325.727151627" Jul 16 21:52:11.337024 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:11.336996 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" event={"ID":"69971137-9cb9-42eb-8b18-5f4e828ff8c1","Type":"ContainerStarted","Data":"a07b603f8c7baf87734f469bc248bbdd0bb565151474fa52408742054b89f3b2"} Jul 16 21:52:12.344864 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:12.344824 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" event={"ID":"69971137-9cb9-42eb-8b18-5f4e828ff8c1","Type":"ContainerStarted","Data":"846466bbdcd4f5cceb1eff4fe36a92914e96a51de63d2d46d8695159864bfad5"} Jul 16 21:52:13.066734 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:13.066695 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:13.066997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:13.066971 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:13.073189 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:13.073158 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:13.353101 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:13.353081 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:52:13.406027 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:13.405994 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb598b595-bcdtr"] Jul 16 21:52:14.355903 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.355871 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" event={"ID":"69971137-9cb9-42eb-8b18-5f4e828ff8c1","Type":"ContainerStarted","Data":"755029c32c404f0d0e39d0d298eb013c65fce873da2877f3ce5346062beefa96"} Jul 16 21:52:14.356295 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.356082 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:52:14.358100 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.358070 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5c8c" event={"ID":"bad1b8fe-5a74-4b1c-a740-06a9c49128c0","Type":"ContainerStarted","Data":"37fb7fe1bae2ad271ef9663822a1ee3d4d07392945a4f5a8c21d1677b1f8b9fd"} Jul 16 21:52:14.361390 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.361370 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a566030-98b2-449f-aaed-1e6caaf3556a","Type":"ContainerStarted","Data":"73083543801d0cc6ecbc362600283f309da5d1738b178ed655af65a096a8716b"} Jul 16 21:52:14.361489 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.361425 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a566030-98b2-449f-aaed-1e6caaf3556a","Type":"ContainerStarted","Data":"497eaaa52ba5ea87ea57a6cb70235b1ba76dc6751261072dcbb7db42577ef3e7"} Jul 16 21:52:14.361489 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.361467 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a566030-98b2-449f-aaed-1e6caaf3556a","Type":"ContainerStarted","Data":"9d6505d2f9bfc9d17b6f15510d9c0ae4a8ab9f692595adaf46085a334cf8359c"} Jul 16 21:52:14.361583 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.361493 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a566030-98b2-449f-aaed-1e6caaf3556a","Type":"ContainerStarted","Data":"23d48e47420c433d760076601e457098afe37bbbfcb0b430cf3db60019cd92f7"} Jul 16 21:52:14.361583 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.361520 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a566030-98b2-449f-aaed-1e6caaf3556a","Type":"ContainerStarted","Data":"b8d9a1aa66b56cca7f4f83f01026be60770c0c4ca605490a277eae8c89af333a"} Jul 16 21:52:14.362788 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.362771 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" Jul 16 21:52:14.382086 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.382036 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7dcbc74488-9r7tl" podStartSLOduration=7.85989554 podStartE2EDuration="18.382023871s" totalImagesPullingTime="10.522128331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:14.380604891 +0000 UTC m=+329.713683947" watchObservedRunningTime="2026-07-16 21:52:14.382023871 +0000 UTC m=+329.715102900" Jul 16 21:52:14.401393 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:14.401329 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n5c8c" podStartSLOduration=35.026326938 podStartE2EDuration="1m1.401316308s" totalImagesPullingTime="26.37498937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:51:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:14.399848879 +0000 UTC m=+329.732927928" watchObservedRunningTime="2026-07-16 21:52:14.401316308 +0000 UTC m=+329.734395338" Jul 16 21:52:15.368434 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:15.368395 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2a566030-98b2-449f-aaed-1e6caaf3556a","Type":"ContainerStarted","Data":"ef71fb0ceca7e0e0ff6753ab3590e79f3e7ae5e775a68a4c39fe60bcfc7d8bb8"} Jul 16 21:52:15.406405 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:15.406355 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=7.693652426 podStartE2EDuration="20.406340263s" totalImagesPullingTime="12.697406524s" totalInitContainerRuntime="15.281313ms" isStatefulPod=false podCreationTimestamp="2026-07-16 21:51:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:15.403813404 +0000 UTC m=+330.736892435" watchObservedRunningTime="2026-07-16 21:52:15.406340263 +0000 UTC m=+330.739419299" Jul 16 21:52:19.792075 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:19.792038 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:52:19.792464 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:19.792090 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:52:23.401221 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:23.401188 2563 generic.go:356] "Generic (PLEG): container finished" podID="b738ba0e-3834-4ff6-9f6b-15d5f79a132d" containerID="e2ef99fae8a709206906cdbfe45ea7ea4575b2fea879f03ea433b3b3b8c81066" exitCode=0 Jul 16 21:52:23.401710 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:23.401249 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" event={"ID":"b738ba0e-3834-4ff6-9f6b-15d5f79a132d","Type":"ContainerDied","Data":"e2ef99fae8a709206906cdbfe45ea7ea4575b2fea879f03ea433b3b3b8c81066"} Jul 16 21:52:23.401710 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:23.401614 2563 scope.go:117] "RemoveContainer" containerID="e2ef99fae8a709206906cdbfe45ea7ea4575b2fea879f03ea433b3b3b8c81066" Jul 16 21:52:24.407681 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:24.407651 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-v9hth" event={"ID":"b738ba0e-3834-4ff6-9f6b-15d5f79a132d","Type":"ContainerStarted","Data":"ca4f89a7b19d8d544a25edfad517725c16a930541725db0bdea2b1fd729c9796"} Jul 16 21:52:26.205058 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.204996 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-58f7c6584-lw844" podUID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" containerName="registry" containerID="cri-o://7179493e96c5454fba3300b5668fa9113233c0be729e43740f8b8469715e9e9c" gracePeriod=30 Jul 16 21:52:26.417614 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.417585 2563 generic.go:356] "Generic (PLEG): container finished" podID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" containerID="7179493e96c5454fba3300b5668fa9113233c0be729e43740f8b8469715e9e9c" exitCode=0 Jul 16 21:52:26.417754 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.417665 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58f7c6584-lw844" event={"ID":"91f0e0b2-de7f-46f9-b428-f0b1e1546653","Type":"ContainerDied","Data":"7179493e96c5454fba3300b5668fa9113233c0be729e43740f8b8469715e9e9c"} Jul 16 21:52:26.479364 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.479313 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:52:26.562152 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562123 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-bound-sa-token\") pod \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " Jul 16 21:52:26.562297 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562182 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkggg\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-kube-api-access-hkggg\") pod \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " Jul 16 21:52:26.562297 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562208 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-installation-pull-secrets\") pod \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " Jul 16 21:52:26.562297 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562259 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") pod \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " Jul 16 21:52:26.562297 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562283 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91f0e0b2-de7f-46f9-b428-f0b1e1546653-ca-trust-extracted\") pod \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " Jul 16 21:52:26.562498 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562315 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-image-registry-private-configuration\") pod \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " Jul 16 21:52:26.562498 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562354 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-certificates\") pod \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " Jul 16 21:52:26.562498 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562429 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-trusted-ca\") pod \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\" (UID: \"91f0e0b2-de7f-46f9-b428-f0b1e1546653\") " Jul 16 21:52:26.562938 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562905 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "91f0e0b2-de7f-46f9-b428-f0b1e1546653" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:52:26.563034 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.562973 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "91f0e0b2-de7f-46f9-b428-f0b1e1546653" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:52:26.564873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.564846 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "91f0e0b2-de7f-46f9-b428-f0b1e1546653" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:52:26.564873 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.564860 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "91f0e0b2-de7f-46f9-b428-f0b1e1546653" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:52:26.565049 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.564888 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-kube-api-access-hkggg" (OuterVolumeSpecName: "kube-api-access-hkggg") pod "91f0e0b2-de7f-46f9-b428-f0b1e1546653" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653"). InnerVolumeSpecName "kube-api-access-hkggg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:52:26.565049 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.564976 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "91f0e0b2-de7f-46f9-b428-f0b1e1546653" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:52:26.565049 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.565014 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "91f0e0b2-de7f-46f9-b428-f0b1e1546653" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:52:26.570768 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.570743 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f0e0b2-de7f-46f9-b428-f0b1e1546653-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "91f0e0b2-de7f-46f9-b428-f0b1e1546653" (UID: "91f0e0b2-de7f-46f9-b428-f0b1e1546653"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:52:26.663550 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.663519 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-trusted-ca\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:26.663550 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.663546 2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-bound-sa-token\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:26.663699 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.663560 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkggg\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-kube-api-access-hkggg\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:26.663699 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.663575 2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-installation-pull-secrets\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:26.663699 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.663586 2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-tls\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:26.663699 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.663598 2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91f0e0b2-de7f-46f9-b428-f0b1e1546653-ca-trust-extracted\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:26.663699 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.663610 2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91f0e0b2-de7f-46f9-b428-f0b1e1546653-image-registry-private-configuration\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:26.663699 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:26.663625 2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91f0e0b2-de7f-46f9-b428-f0b1e1546653-registry-certificates\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:27.422747 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:27.422725 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58f7c6584-lw844" Jul 16 21:52:27.423232 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:27.422720 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58f7c6584-lw844" event={"ID":"91f0e0b2-de7f-46f9-b428-f0b1e1546653","Type":"ContainerDied","Data":"442bee796dbe600f24af366b682a17ea01cfd898fd4c58061d4ef9768684a6d8"} Jul 16 21:52:27.423232 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:27.422874 2563 scope.go:117] "RemoveContainer" containerID="7179493e96c5454fba3300b5668fa9113233c0be729e43740f8b8469715e9e9c" Jul 16 21:52:27.443021 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:27.443000 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58f7c6584-lw844"] Jul 16 21:52:27.445387 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:27.445366 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-58f7c6584-lw844"] Jul 16 21:52:28.428926 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:28.428842 2563 generic.go:356] "Generic (PLEG): container finished" podID="81e44120-3c18-4e07-ba65-7ce68c3255e8" containerID="f472c6e97c4e7312d28c657cc9b389ff35c3094e6da6d3186a9757cfd4381408" exitCode=0 Jul 16 21:52:28.429408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:28.428916 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-prwdk" event={"ID":"81e44120-3c18-4e07-ba65-7ce68c3255e8","Type":"ContainerDied","Data":"f472c6e97c4e7312d28c657cc9b389ff35c3094e6da6d3186a9757cfd4381408"} Jul 16 21:52:28.429408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:28.429363 2563 scope.go:117] "RemoveContainer" containerID="f472c6e97c4e7312d28c657cc9b389ff35c3094e6da6d3186a9757cfd4381408" Jul 16 21:52:29.241058 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:29.241024 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" path="/var/lib/kubelet/pods/91f0e0b2-de7f-46f9-b428-f0b1e1546653/volumes" Jul 16 21:52:29.262472 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:29.262447 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-544c55cfbb-t62g9_838fa3a2-b821-45b7-9678-a60fa9e797f2/router/0.log" Jul 16 21:52:29.303834 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:29.303812 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f22kb_d366d608-8179-4e70-8529-6dce72b0b665/serve-healthcheck-canary/0.log" Jul 16 21:52:29.439556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:29.439520 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-prwdk" event={"ID":"81e44120-3c18-4e07-ba65-7ce68c3255e8","Type":"ContainerStarted","Data":"f48b3e8aa4d54a0a2dedeffcfc537a9c8c3f9388ebf3527ef21cac290c7c355c"} Jul 16 21:52:38.427815 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.427733 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bb598b595-bcdtr" podUID="3be744f6-6352-476f-be6b-825aa2e559ff" containerName="console" containerID="cri-o://5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc" gracePeriod=15 Jul 16 21:52:38.705159 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.705075 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:52:38.772349 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772320 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-oauth-serving-cert\") pod \"3be744f6-6352-476f-be6b-825aa2e559ff\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " Jul 16 21:52:38.772349 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772351 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-console-config\") pod \"3be744f6-6352-476f-be6b-825aa2e559ff\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " Jul 16 21:52:38.772575 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772369 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-service-ca\") pod \"3be744f6-6352-476f-be6b-825aa2e559ff\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " Jul 16 21:52:38.772575 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772397 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vdgn\" (UniqueName: \"kubernetes.io/projected/3be744f6-6352-476f-be6b-825aa2e559ff-kube-api-access-9vdgn\") pod \"3be744f6-6352-476f-be6b-825aa2e559ff\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " Jul 16 21:52:38.772575 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772414 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-serving-cert\") pod \"3be744f6-6352-476f-be6b-825aa2e559ff\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " Jul 16 21:52:38.772575 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772437 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-oauth-config\") pod \"3be744f6-6352-476f-be6b-825aa2e559ff\" (UID: \"3be744f6-6352-476f-be6b-825aa2e559ff\") " Jul 16 21:52:38.772842 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772818 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3be744f6-6352-476f-be6b-825aa2e559ff" (UID: "3be744f6-6352-476f-be6b-825aa2e559ff"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:52:38.772905 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772833 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-service-ca" (OuterVolumeSpecName: "service-ca") pod "3be744f6-6352-476f-be6b-825aa2e559ff" (UID: "3be744f6-6352-476f-be6b-825aa2e559ff"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:52:38.772905 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.772846 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-console-config" (OuterVolumeSpecName: "console-config") pod "3be744f6-6352-476f-be6b-825aa2e559ff" (UID: "3be744f6-6352-476f-be6b-825aa2e559ff"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:52:38.774739 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.774719 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3be744f6-6352-476f-be6b-825aa2e559ff" (UID: "3be744f6-6352-476f-be6b-825aa2e559ff"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:52:38.774801 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.774735 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3be744f6-6352-476f-be6b-825aa2e559ff" (UID: "3be744f6-6352-476f-be6b-825aa2e559ff"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:52:38.774858 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.774837 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3be744f6-6352-476f-be6b-825aa2e559ff-kube-api-access-9vdgn" (OuterVolumeSpecName: "kube-api-access-9vdgn") pod "3be744f6-6352-476f-be6b-825aa2e559ff" (UID: "3be744f6-6352-476f-be6b-825aa2e559ff"). InnerVolumeSpecName "kube-api-access-9vdgn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:52:38.873315 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.873290 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-oauth-serving-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:38.873315 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.873312 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-console-config\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:38.873444 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.873322 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3be744f6-6352-476f-be6b-825aa2e559ff-service-ca\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:38.873444 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.873330 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vdgn\" (UniqueName: \"kubernetes.io/projected/3be744f6-6352-476f-be6b-825aa2e559ff-kube-api-access-9vdgn\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:38.873444 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.873338 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-serving-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:38.873444 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:38.873346 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3be744f6-6352-476f-be6b-825aa2e559ff-console-oauth-config\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:52:39.481776 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.481745 2563 generic.go:356] "Generic (PLEG): container finished" podID="3be744f6-6352-476f-be6b-825aa2e559ff" containerID="5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc" exitCode=0 Jul 16 21:52:39.482276 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.481790 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb598b595-bcdtr" Jul 16 21:52:39.482276 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.481824 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb598b595-bcdtr" event={"ID":"3be744f6-6352-476f-be6b-825aa2e559ff","Type":"ContainerDied","Data":"5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc"} Jul 16 21:52:39.482276 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.481882 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb598b595-bcdtr" event={"ID":"3be744f6-6352-476f-be6b-825aa2e559ff","Type":"ContainerDied","Data":"bd74338ac1308a449cf19e1655d3b2d9d765d7f1c2a9f4a63a70482aa53c3a4a"} Jul 16 21:52:39.482276 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.481902 2563 scope.go:117] "RemoveContainer" containerID="5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc" Jul 16 21:52:39.490575 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.490561 2563 scope.go:117] "RemoveContainer" containerID="5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc" Jul 16 21:52:39.490833 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:52:39.490811 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc\": container with ID starting with 5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc not found: ID does not exist" containerID="5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc" Jul 16 21:52:39.490906 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.490850 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc"} err="failed to get container status \"5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc\": rpc error: code = NotFound desc = could not find container \"5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc\": container with ID starting with 5b68f4daf6f8a0f1b10db9a4cb9280bc644811b6ecd1364f20ebbc367cd274bc not found: ID does not exist" Jul 16 21:52:39.508015 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.507961 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb598b595-bcdtr"] Jul 16 21:52:39.514684 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.514662 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bb598b595-bcdtr"] Jul 16 21:52:39.798091 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.798026 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:52:39.802005 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:39.801971 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-c8c4fdcb9-fhg9d" Jul 16 21:52:41.239568 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:41.239540 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3be744f6-6352-476f-be6b-825aa2e559ff" path="/var/lib/kubelet/pods/3be744f6-6352-476f-be6b-825aa2e559ff/volumes" Jul 16 21:52:58.944179 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:58.944118 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:52:58.946516 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:58.946490 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ef310c53-4c7a-4331-91b9-6e7a2bb27841-metrics-certs\") pod \"network-metrics-daemon-xm229\" (UID: \"ef310c53-4c7a-4331-91b9-6e7a2bb27841\") " pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:52:59.238900 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:59.238823 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gkkkc\"" Jul 16 21:52:59.245920 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:59.245901 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xm229" Jul 16 21:52:59.423410 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:59.423389 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xm229"] Jul 16 21:52:59.426108 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:52:59.426082 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef310c53_4c7a_4331_91b9_6e7a2bb27841.slice/crio-64a3c5d729310a14a4c91de8be46957efeed3a8d35bfc1801a5f19d2a211f7ea WatchSource:0}: Error finding container 64a3c5d729310a14a4c91de8be46957efeed3a8d35bfc1801a5f19d2a211f7ea: Status 404 returned error can't find the container with id 64a3c5d729310a14a4c91de8be46957efeed3a8d35bfc1801a5f19d2a211f7ea Jul 16 21:52:59.571659 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:52:59.571597 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm229" event={"ID":"ef310c53-4c7a-4331-91b9-6e7a2bb27841","Type":"ContainerStarted","Data":"64a3c5d729310a14a4c91de8be46957efeed3a8d35bfc1801a5f19d2a211f7ea"} Jul 16 21:53:00.580119 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:00.580091 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm229" event={"ID":"ef310c53-4c7a-4331-91b9-6e7a2bb27841","Type":"ContainerStarted","Data":"74c388d1fcadef830b0353c3ccd814444f0b096d0a05f8e820a66795cc176c19"} Jul 16 21:53:01.585302 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:01.585265 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xm229" event={"ID":"ef310c53-4c7a-4331-91b9-6e7a2bb27841","Type":"ContainerStarted","Data":"7b6342c9a9b30bbc4f7d99f4fe33d8dcee2d0f1ad4fea82ec8d848e087ad0392"} Jul 16 21:53:01.612193 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:01.612108 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xm229" podStartSLOduration=375.543556042 podStartE2EDuration="6m16.612088541s" totalImagesPullingTime="1.068532499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:46:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:53:01.603724961 +0000 UTC m=+376.936804000" watchObservedRunningTime="2026-07-16 21:53:01.612088541 +0000 UTC m=+376.945167571" Jul 16 21:53:19.207504 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.207467 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn"] Jul 16 21:53:19.207985 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.207840 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" containerName="registry" Jul 16 21:53:19.207985 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.207854 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" containerName="registry" Jul 16 21:53:19.207985 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.207875 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3be744f6-6352-476f-be6b-825aa2e559ff" containerName="console" Jul 16 21:53:19.207985 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.207881 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3be744f6-6352-476f-be6b-825aa2e559ff" containerName="console" Jul 16 21:53:19.207985 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.207938 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f0e0b2-de7f-46f9-b428-f0b1e1546653" containerName="registry" Jul 16 21:53:19.207985 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.207945 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3be744f6-6352-476f-be6b-825aa2e559ff" containerName="console" Jul 16 21:53:19.210364 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.210346 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.213546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.213529 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 16 21:53:19.213716 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.213671 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 16 21:53:19.213716 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.213705 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 16 21:53:19.213854 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.213705 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 16 21:53:19.213854 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.213744 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 16 21:53:19.213854 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.213825 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fgpnh\"" Jul 16 21:53:19.219936 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.219911 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 16 21:53:19.226327 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.226300 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn"] Jul 16 21:53:19.298616 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.298582 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-secret-telemeter-client\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.298616 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.298616 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-federate-client-tls\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.298805 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.298654 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-serving-certs-ca-bundle\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.298805 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.298688 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-metrics-client-ca\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.298805 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.298741 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.298900 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.298837 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-telemeter-client-tls\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.298900 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.298874 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b65ln\" (UniqueName: \"kubernetes.io/projected/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-kube-api-access-b65ln\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.298963 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.298910 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.400024 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.399999 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.400189 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.400055 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-telemeter-client-tls\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.400189 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.400073 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b65ln\" (UniqueName: \"kubernetes.io/projected/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-kube-api-access-b65ln\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.400189 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.400104 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.400189 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.400178 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-secret-telemeter-client\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.400355 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.400203 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-federate-client-tls\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.400355 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.400239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-serving-certs-ca-bundle\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.400355 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.400261 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-metrics-client-ca\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.401068 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.401043 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.401209 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.401095 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-serving-certs-ca-bundle\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.401209 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.401099 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-metrics-client-ca\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.402673 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.402648 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-telemeter-client-tls\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.402785 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.402699 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-federate-client-tls\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.402978 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.402959 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.403061 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.403042 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-secret-telemeter-client\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.409415 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.409392 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b65ln\" (UniqueName: \"kubernetes.io/projected/392f5bf8-1c67-41fb-aed0-c8040f9fefe2-kube-api-access-b65ln\") pod \"telemeter-client-554b5d6ff9-r7ncn\" (UID: \"392f5bf8-1c67-41fb-aed0-c8040f9fefe2\") " pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.519696 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.519664 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" Jul 16 21:53:19.653455 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:19.653383 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn"] Jul 16 21:53:19.656556 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:53:19.656526 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392f5bf8_1c67_41fb_aed0_c8040f9fefe2.slice/crio-2e506e224cc1585410279b41a7dc64cb8a79d3e4238a3d60a916fc69d03135ed WatchSource:0}: Error finding container 2e506e224cc1585410279b41a7dc64cb8a79d3e4238a3d60a916fc69d03135ed: Status 404 returned error can't find the container with id 2e506e224cc1585410279b41a7dc64cb8a79d3e4238a3d60a916fc69d03135ed Jul 16 21:53:20.660228 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:20.660183 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" event={"ID":"392f5bf8-1c67-41fb-aed0-c8040f9fefe2","Type":"ContainerStarted","Data":"2e506e224cc1585410279b41a7dc64cb8a79d3e4238a3d60a916fc69d03135ed"} Jul 16 21:53:21.666763 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:21.666727 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" event={"ID":"392f5bf8-1c67-41fb-aed0-c8040f9fefe2","Type":"ContainerStarted","Data":"9e35b0c4df0e84592e789b73994c4238a8d7cc079fa04eaefc59a509afc525aa"} Jul 16 21:53:21.667247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:21.666793 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" event={"ID":"392f5bf8-1c67-41fb-aed0-c8040f9fefe2","Type":"ContainerStarted","Data":"0412ab0c9765a959603ea8427132bda3f0ed6a3d72d97c7c2ef65a88c7cda2a8"} Jul 16 21:53:21.667247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:21.666820 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" event={"ID":"392f5bf8-1c67-41fb-aed0-c8040f9fefe2","Type":"ContainerStarted","Data":"f94433c6571aaa4395edda611c2eff9a83da678245d9a812bd504f79f65b935e"} Jul 16 21:53:21.692137 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:21.692082 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-554b5d6ff9-r7ncn" podStartSLOduration=1.165824389 podStartE2EDuration="2.692066554s" totalImagesPullingTime="1.526242165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:53:21.689381288 +0000 UTC m=+397.022460339" watchObservedRunningTime="2026-07-16 21:53:21.692066554 +0000 UTC m=+397.025145613" Jul 16 21:53:22.465419 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.465382 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54874b6555-272jv"] Jul 16 21:53:22.468060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.468037 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.482171 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.482128 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54874b6555-272jv"] Jul 16 21:53:22.629182 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.629134 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-oauth-config\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.629182 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.629186 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-service-ca\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.629364 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.629212 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-serving-cert\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.629364 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.629265 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjhkb\" (UniqueName: \"kubernetes.io/projected/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-kube-api-access-hjhkb\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.629364 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.629317 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-oauth-serving-cert\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.629364 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.629358 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-trusted-ca-bundle\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.629506 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.629380 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-config\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.730401 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.730322 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjhkb\" (UniqueName: \"kubernetes.io/projected/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-kube-api-access-hjhkb\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.730401 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.730357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-oauth-serving-cert\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.730401 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.730384 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-trusted-ca-bundle\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.730987 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.730405 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-config\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.730987 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.730433 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-oauth-config\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.730987 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.730451 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-service-ca\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.730987 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.730474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-serving-cert\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.731241 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.731130 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-config\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.731241 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.731167 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-oauth-serving-cert\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.731348 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.731289 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-service-ca\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.731348 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.731323 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-trusted-ca-bundle\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.732967 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.732947 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-serving-cert\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.732967 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.732960 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-oauth-config\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.740081 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.740060 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjhkb\" (UniqueName: \"kubernetes.io/projected/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-kube-api-access-hjhkb\") pod \"console-54874b6555-272jv\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.779068 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.779045 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:22.913312 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:22.913286 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54874b6555-272jv"] Jul 16 21:53:22.915155 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:53:22.915114 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5bc5cce_3c34_4eee_ada7_3216f5d0a4f2.slice/crio-fc98e4e5a44c417be21f5bece673f2f0357a6564597c3073b3e7f1fc9c7a1ab2 WatchSource:0}: Error finding container fc98e4e5a44c417be21f5bece673f2f0357a6564597c3073b3e7f1fc9c7a1ab2: Status 404 returned error can't find the container with id fc98e4e5a44c417be21f5bece673f2f0357a6564597c3073b3e7f1fc9c7a1ab2 Jul 16 21:53:23.676441 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:23.676402 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54874b6555-272jv" event={"ID":"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2","Type":"ContainerStarted","Data":"9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df"} Jul 16 21:53:23.676603 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:23.676455 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54874b6555-272jv" event={"ID":"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2","Type":"ContainerStarted","Data":"fc98e4e5a44c417be21f5bece673f2f0357a6564597c3073b3e7f1fc9c7a1ab2"} Jul 16 21:53:23.696482 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:23.696436 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54874b6555-272jv" podStartSLOduration=1.696421159 podStartE2EDuration="1.696421159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:53:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:53:23.695195938 +0000 UTC m=+399.028274987" watchObservedRunningTime="2026-07-16 21:53:23.696421159 +0000 UTC m=+399.029500188" Jul 16 21:53:30.986841 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:30.986781 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54874b6555-272jv"] Jul 16 21:53:32.779557 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:32.779513 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:41.567151 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:41.567116 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bfd64b99b-ntf8t"] Jul 16 21:53:56.006547 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.006500 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54874b6555-272jv" podUID="a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" containerName="console" containerID="cri-o://9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df" gracePeriod=15 Jul 16 21:53:56.245618 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.245595 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:56.319399 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319335 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-config\") pod \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " Jul 16 21:53:56.319399 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319398 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-service-ca\") pod \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " Jul 16 21:53:56.319580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319417 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-trusted-ca-bundle\") pod \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " Jul 16 21:53:56.319580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319436 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjhkb\" (UniqueName: \"kubernetes.io/projected/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-kube-api-access-hjhkb\") pod \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " Jul 16 21:53:56.319580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319468 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-oauth-config\") pod \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " Jul 16 21:53:56.319580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319494 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-serving-cert\") pod \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " Jul 16 21:53:56.319580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319546 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-oauth-serving-cert\") pod \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\" (UID: \"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2\") " Jul 16 21:53:56.319821 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319785 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-config" (OuterVolumeSpecName: "console-config") pod "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" (UID: "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:53:56.319976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319909 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-service-ca" (OuterVolumeSpecName: "service-ca") pod "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" (UID: "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:53:56.320086 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.319940 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" (UID: "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:53:56.320086 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.320046 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" (UID: "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:53:56.321596 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.321576 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-kube-api-access-hjhkb" (OuterVolumeSpecName: "kube-api-access-hjhkb") pod "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" (UID: "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2"). InnerVolumeSpecName "kube-api-access-hjhkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:53:56.322085 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.322063 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" (UID: "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:53:56.322175 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.322053 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" (UID: "a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:53:56.420468 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.420441 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-config\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:53:56.420468 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.420465 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-service-ca\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:53:56.420468 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.420473 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-trusted-ca-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:53:56.420649 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.420483 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjhkb\" (UniqueName: \"kubernetes.io/projected/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-kube-api-access-hjhkb\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:53:56.420649 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.420492 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-oauth-config\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:53:56.420649 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.420500 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-console-serving-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:53:56.420649 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.420509 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2-oauth-serving-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:53:56.822892 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.822860 2563 generic.go:356] "Generic (PLEG): container finished" podID="a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" containerID="9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df" exitCode=0 Jul 16 21:53:56.823080 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.822902 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54874b6555-272jv" Jul 16 21:53:56.823080 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.822933 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54874b6555-272jv" event={"ID":"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2","Type":"ContainerDied","Data":"9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df"} Jul 16 21:53:56.823080 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.822982 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54874b6555-272jv" event={"ID":"a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2","Type":"ContainerDied","Data":"fc98e4e5a44c417be21f5bece673f2f0357a6564597c3073b3e7f1fc9c7a1ab2"} Jul 16 21:53:56.823080 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.823008 2563 scope.go:117] "RemoveContainer" containerID="9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df" Jul 16 21:53:56.831610 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.831592 2563 scope.go:117] "RemoveContainer" containerID="9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df" Jul 16 21:53:56.831853 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:53:56.831833 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df\": container with ID starting with 9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df not found: ID does not exist" containerID="9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df" Jul 16 21:53:56.831898 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.831861 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df"} err="failed to get container status \"9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df\": rpc error: code = NotFound desc = could not find container \"9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df\": container with ID starting with 9c0078a3b01f342de9278df3286d2a9de2cf74caeeda2a7463b7d0e86961e4df not found: ID does not exist" Jul 16 21:53:56.857862 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.857840 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54874b6555-272jv"] Jul 16 21:53:56.872049 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:56.872027 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54874b6555-272jv"] Jul 16 21:53:57.243799 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:53:57.241126 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" path="/var/lib/kubelet/pods/a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2/volumes" Jul 16 21:54:06.590755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.590695 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bfd64b99b-ntf8t" podUID="4c660ea6-ed99-41db-92c9-326bb5814f76" containerName="console" containerID="cri-o://c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456" gracePeriod=15 Jul 16 21:54:06.833048 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.833024 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:54:06.868821 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.868794 2563 generic.go:356] "Generic (PLEG): container finished" podID="4c660ea6-ed99-41db-92c9-326bb5814f76" containerID="c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456" exitCode=0 Jul 16 21:54:06.868947 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.868843 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bfd64b99b-ntf8t" Jul 16 21:54:06.868947 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.868891 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bfd64b99b-ntf8t" event={"ID":"4c660ea6-ed99-41db-92c9-326bb5814f76","Type":"ContainerDied","Data":"c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456"} Jul 16 21:54:06.868947 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.868940 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bfd64b99b-ntf8t" event={"ID":"4c660ea6-ed99-41db-92c9-326bb5814f76","Type":"ContainerDied","Data":"3f6c5dfdafe40465c3b7e2ee5db552f8d40f292b06b16543d0a69d375eaff972"} Jul 16 21:54:06.869049 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.868964 2563 scope.go:117] "RemoveContainer" containerID="c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456" Jul 16 21:54:06.876727 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.876710 2563 scope.go:117] "RemoveContainer" containerID="c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456" Jul 16 21:54:06.876998 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:54:06.876976 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456\": container with ID starting with c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456 not found: ID does not exist" containerID="c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456" Jul 16 21:54:06.877080 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:06.877011 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456"} err="failed to get container status \"c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456\": rpc error: code = NotFound desc = could not find container \"c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456\": container with ID starting with c54cf0c7b7579e518613d196f123c61812155b21203783360d816232736d2456 not found: ID does not exist" Jul 16 21:54:07.011153 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011114 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w4kd\" (UniqueName: \"kubernetes.io/projected/4c660ea6-ed99-41db-92c9-326bb5814f76-kube-api-access-9w4kd\") pod \"4c660ea6-ed99-41db-92c9-326bb5814f76\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " Jul 16 21:54:07.011285 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011181 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-console-config\") pod \"4c660ea6-ed99-41db-92c9-326bb5814f76\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " Jul 16 21:54:07.011285 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011233 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-service-ca\") pod \"4c660ea6-ed99-41db-92c9-326bb5814f76\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " Jul 16 21:54:07.011285 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011269 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-trusted-ca-bundle\") pod \"4c660ea6-ed99-41db-92c9-326bb5814f76\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " Jul 16 21:54:07.011392 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011330 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-oauth-serving-cert\") pod \"4c660ea6-ed99-41db-92c9-326bb5814f76\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " Jul 16 21:54:07.011392 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011368 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-oauth-config\") pod \"4c660ea6-ed99-41db-92c9-326bb5814f76\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " Jul 16 21:54:07.011495 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011391 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-serving-cert\") pod \"4c660ea6-ed99-41db-92c9-326bb5814f76\" (UID: \"4c660ea6-ed99-41db-92c9-326bb5814f76\") " Jul 16 21:54:07.011705 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011679 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-console-config" (OuterVolumeSpecName: "console-config") pod "4c660ea6-ed99-41db-92c9-326bb5814f76" (UID: "4c660ea6-ed99-41db-92c9-326bb5814f76"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:54:07.011806 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011697 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-service-ca" (OuterVolumeSpecName: "service-ca") pod "4c660ea6-ed99-41db-92c9-326bb5814f76" (UID: "4c660ea6-ed99-41db-92c9-326bb5814f76"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:54:07.011806 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011709 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4c660ea6-ed99-41db-92c9-326bb5814f76" (UID: "4c660ea6-ed99-41db-92c9-326bb5814f76"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:54:07.011806 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.011724 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4c660ea6-ed99-41db-92c9-326bb5814f76" (UID: "4c660ea6-ed99-41db-92c9-326bb5814f76"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:54:07.013209 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.013181 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c660ea6-ed99-41db-92c9-326bb5814f76-kube-api-access-9w4kd" (OuterVolumeSpecName: "kube-api-access-9w4kd") pod "4c660ea6-ed99-41db-92c9-326bb5814f76" (UID: "4c660ea6-ed99-41db-92c9-326bb5814f76"). InnerVolumeSpecName "kube-api-access-9w4kd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:54:07.013383 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.013363 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4c660ea6-ed99-41db-92c9-326bb5814f76" (UID: "4c660ea6-ed99-41db-92c9-326bb5814f76"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:54:07.013425 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.013394 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4c660ea6-ed99-41db-92c9-326bb5814f76" (UID: "4c660ea6-ed99-41db-92c9-326bb5814f76"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:54:07.112647 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.112625 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-service-ca\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:07.112647 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.112646 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-trusted-ca-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:07.112777 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.112655 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-oauth-serving-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:07.112777 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.112663 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-oauth-config\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:07.112777 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.112671 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4c660ea6-ed99-41db-92c9-326bb5814f76-console-serving-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:07.112777 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.112680 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w4kd\" (UniqueName: \"kubernetes.io/projected/4c660ea6-ed99-41db-92c9-326bb5814f76-kube-api-access-9w4kd\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:07.112777 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.112689 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4c660ea6-ed99-41db-92c9-326bb5814f76-console-config\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:07.204203 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.204136 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bfd64b99b-ntf8t"] Jul 16 21:54:07.212976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.212950 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bfd64b99b-ntf8t"] Jul 16 21:54:07.239074 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:07.239045 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c660ea6-ed99-41db-92c9-326bb5814f76" path="/var/lib/kubelet/pods/4c660ea6-ed99-41db-92c9-326bb5814f76/volumes" Jul 16 21:54:18.027107 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.027071 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn"] Jul 16 21:54:18.027546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.027450 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" containerName="console" Jul 16 21:54:18.027546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.027463 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" containerName="console" Jul 16 21:54:18.027546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.027477 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c660ea6-ed99-41db-92c9-326bb5814f76" containerName="console" Jul 16 21:54:18.027546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.027483 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c660ea6-ed99-41db-92c9-326bb5814f76" containerName="console" Jul 16 21:54:18.027546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.027531 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c660ea6-ed99-41db-92c9-326bb5814f76" containerName="console" Jul 16 21:54:18.027546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.027543 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5bc5cce-3c34-4eee-ada7-3216f5d0a4f2" containerName="console" Jul 16 21:54:18.030728 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.030711 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.033728 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.033707 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tdcn5\"" Jul 16 21:54:18.033868 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.033710 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 21:54:18.033868 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.033710 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 21:54:18.039701 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.039676 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn"] Jul 16 21:54:18.096747 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.096720 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqtp4\" (UniqueName: \"kubernetes.io/projected/63cdb4de-2725-4f58-95fe-a65a687a9f51-kube-api-access-xqtp4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.096855 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.096763 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.096855 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.096833 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.197904 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.197874 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.198043 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.197932 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqtp4\" (UniqueName: \"kubernetes.io/projected/63cdb4de-2725-4f58-95fe-a65a687a9f51-kube-api-access-xqtp4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.198043 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.197977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.198272 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.198251 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.198315 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.198287 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.207726 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.207704 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqtp4\" (UniqueName: \"kubernetes.io/projected/63cdb4de-2725-4f58-95fe-a65a687a9f51-kube-api-access-xqtp4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.340272 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.340197 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:18.489870 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.489846 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn"] Jul 16 21:54:18.492498 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:54:18.492472 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63cdb4de_2725_4f58_95fe_a65a687a9f51.slice/crio-0be1cd8c70ad1d553333dfde413c4911d41e17e3a4bb0ad616423f7281de0540 WatchSource:0}: Error finding container 0be1cd8c70ad1d553333dfde413c4911d41e17e3a4bb0ad616423f7281de0540: Status 404 returned error can't find the container with id 0be1cd8c70ad1d553333dfde413c4911d41e17e3a4bb0ad616423f7281de0540 Jul 16 21:54:18.925162 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:18.925109 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" event={"ID":"63cdb4de-2725-4f58-95fe-a65a687a9f51","Type":"ContainerStarted","Data":"0be1cd8c70ad1d553333dfde413c4911d41e17e3a4bb0ad616423f7281de0540"} Jul 16 21:54:24.961475 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:24.961441 2563 generic.go:356] "Generic (PLEG): container finished" podID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerID="2cd3a5830e7ded5f69068ded2462bc028d7a3594cacb2d07da3435231420e20d" exitCode=0 Jul 16 21:54:24.961910 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:24.961529 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" event={"ID":"63cdb4de-2725-4f58-95fe-a65a687a9f51","Type":"ContainerDied","Data":"2cd3a5830e7ded5f69068ded2462bc028d7a3594cacb2d07da3435231420e20d"} Jul 16 21:54:27.977067 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:27.977038 2563 generic.go:356] "Generic (PLEG): container finished" podID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerID="97dcbee78bd8cf2d9eae9a6cf988963efb3de39c288ae4d7cc4cf1f82090160d" exitCode=0 Jul 16 21:54:27.977465 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:27.977087 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" event={"ID":"63cdb4de-2725-4f58-95fe-a65a687a9f51","Type":"ContainerDied","Data":"97dcbee78bd8cf2d9eae9a6cf988963efb3de39c288ae4d7cc4cf1f82090160d"} Jul 16 21:54:35.013973 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:35.013940 2563 generic.go:356] "Generic (PLEG): container finished" podID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerID="3562648ed50e68cd1fb7af7660a7be1dd36fa302ea9c4b58ddeb5241d60e5e44" exitCode=0 Jul 16 21:54:35.014376 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:35.014027 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" event={"ID":"63cdb4de-2725-4f58-95fe-a65a687a9f51","Type":"ContainerDied","Data":"3562648ed50e68cd1fb7af7660a7be1dd36fa302ea9c4b58ddeb5241d60e5e44"} Jul 16 21:54:36.153103 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.153081 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:36.263827 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.263795 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqtp4\" (UniqueName: \"kubernetes.io/projected/63cdb4de-2725-4f58-95fe-a65a687a9f51-kube-api-access-xqtp4\") pod \"63cdb4de-2725-4f58-95fe-a65a687a9f51\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " Jul 16 21:54:36.263979 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.263923 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-bundle\") pod \"63cdb4de-2725-4f58-95fe-a65a687a9f51\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " Jul 16 21:54:36.263979 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.263959 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-util\") pod \"63cdb4de-2725-4f58-95fe-a65a687a9f51\" (UID: \"63cdb4de-2725-4f58-95fe-a65a687a9f51\") " Jul 16 21:54:36.264691 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.264665 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-bundle" (OuterVolumeSpecName: "bundle") pod "63cdb4de-2725-4f58-95fe-a65a687a9f51" (UID: "63cdb4de-2725-4f58-95fe-a65a687a9f51"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:54:36.265872 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.265853 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63cdb4de-2725-4f58-95fe-a65a687a9f51-kube-api-access-xqtp4" (OuterVolumeSpecName: "kube-api-access-xqtp4") pod "63cdb4de-2725-4f58-95fe-a65a687a9f51" (UID: "63cdb4de-2725-4f58-95fe-a65a687a9f51"). InnerVolumeSpecName "kube-api-access-xqtp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:54:36.269417 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.269373 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-util" (OuterVolumeSpecName: "util") pod "63cdb4de-2725-4f58-95fe-a65a687a9f51" (UID: "63cdb4de-2725-4f58-95fe-a65a687a9f51"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:54:36.365525 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.365458 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:36.365525 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.365485 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/63cdb4de-2725-4f58-95fe-a65a687a9f51-util\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:36.365696 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:36.365573 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqtp4\" (UniqueName: \"kubernetes.io/projected/63cdb4de-2725-4f58-95fe-a65a687a9f51-kube-api-access-xqtp4\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:37.023707 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:37.023677 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" Jul 16 21:54:37.023925 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:37.023679 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540lzzzn" event={"ID":"63cdb4de-2725-4f58-95fe-a65a687a9f51","Type":"ContainerDied","Data":"0be1cd8c70ad1d553333dfde413c4911d41e17e3a4bb0ad616423f7281de0540"} Jul 16 21:54:37.023925 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:37.023793 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0be1cd8c70ad1d553333dfde413c4911d41e17e3a4bb0ad616423f7281de0540" Jul 16 21:54:40.973936 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:40.973907 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn"] Jul 16 21:54:40.974430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:40.974246 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerName="pull" Jul 16 21:54:40.974430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:40.974258 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerName="pull" Jul 16 21:54:40.974430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:40.974266 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerName="extract" Jul 16 21:54:40.974430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:40.974272 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerName="extract" Jul 16 21:54:40.974430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:40.974283 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerName="util" Jul 16 21:54:40.974430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:40.974288 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerName="util" Jul 16 21:54:40.974430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:40.974344 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="63cdb4de-2725-4f58-95fe-a65a687a9f51" containerName="extract" Jul 16 21:54:41.015389 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.015361 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn"] Jul 16 21:54:41.015522 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.015451 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" Jul 16 21:54:41.018364 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.018340 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7tdtw\"" Jul 16 21:54:41.018524 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.018504 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:54:41.018620 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.018605 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 21:54:41.105427 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.105396 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66bca124-1900-42f1-8375-306068d1fe02-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtdqn\" (UID: \"66bca124-1900-42f1-8375-306068d1fe02\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" Jul 16 21:54:41.105555 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.105434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khsvr\" (UniqueName: \"kubernetes.io/projected/66bca124-1900-42f1-8375-306068d1fe02-kube-api-access-khsvr\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtdqn\" (UID: \"66bca124-1900-42f1-8375-306068d1fe02\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" Jul 16 21:54:41.206650 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.206621 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66bca124-1900-42f1-8375-306068d1fe02-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtdqn\" (UID: \"66bca124-1900-42f1-8375-306068d1fe02\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" Jul 16 21:54:41.206760 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.206660 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khsvr\" (UniqueName: \"kubernetes.io/projected/66bca124-1900-42f1-8375-306068d1fe02-kube-api-access-khsvr\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtdqn\" (UID: \"66bca124-1900-42f1-8375-306068d1fe02\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" Jul 16 21:54:41.206992 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.206974 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66bca124-1900-42f1-8375-306068d1fe02-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtdqn\" (UID: \"66bca124-1900-42f1-8375-306068d1fe02\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" Jul 16 21:54:41.214972 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.214944 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khsvr\" (UniqueName: \"kubernetes.io/projected/66bca124-1900-42f1-8375-306068d1fe02-kube-api-access-khsvr\") pod \"cert-manager-operator-controller-manager-55c444bf86-vtdqn\" (UID: \"66bca124-1900-42f1-8375-306068d1fe02\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" Jul 16 21:54:41.325221 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.325155 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" Jul 16 21:54:41.477826 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:41.477799 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn"] Jul 16 21:54:41.481756 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:54:41.481718 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66bca124_1900_42f1_8375_306068d1fe02.slice/crio-ed6b6f410a60544d111387d8c05a20c0533d25c1abf533f86d1b8428283f92f7 WatchSource:0}: Error finding container ed6b6f410a60544d111387d8c05a20c0533d25c1abf533f86d1b8428283f92f7: Status 404 returned error can't find the container with id ed6b6f410a60544d111387d8c05a20c0533d25c1abf533f86d1b8428283f92f7 Jul 16 21:54:42.049644 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:42.049610 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" event={"ID":"66bca124-1900-42f1-8375-306068d1fe02","Type":"ContainerStarted","Data":"ed6b6f410a60544d111387d8c05a20c0533d25c1abf533f86d1b8428283f92f7"} Jul 16 21:54:44.067313 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:44.067277 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" event={"ID":"66bca124-1900-42f1-8375-306068d1fe02","Type":"ContainerStarted","Data":"786d3ec0ea2b49988f03013ef7041a8fff1c1cc4ed35bde9583dd040e5019311"} Jul 16 21:54:44.102958 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:44.102901 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vtdqn" podStartSLOduration=1.7579589150000001 podStartE2EDuration="4.102881509s" totalImagesPullingTime="2.344922594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:44.099108482 +0000 UTC m=+479.432187517" watchObservedRunningTime="2026-07-16 21:54:44.102881509 +0000 UTC m=+479.435960540" Jul 16 21:54:45.116462 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.116434 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mm5q6"] Jul 16 21:54:45.119194 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.119177 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:45.123239 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.123212 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 21:54:45.123755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.123738 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 21:54:45.135971 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.135952 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mm5q6"] Jul 16 21:54:45.170275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.170249 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr"] Jul 16 21:54:45.172564 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.172548 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.177883 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.177863 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 21:54:45.177985 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.177920 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tdcn5\"" Jul 16 21:54:45.178056 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.177928 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 21:54:45.193782 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.193755 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr"] Jul 16 21:54:45.242592 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.242567 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jddft\" (UniqueName: \"kubernetes.io/projected/de99ab8c-8910-4235-bbe5-eb71b825d69e-kube-api-access-jddft\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.242725 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.242610 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.242801 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.242731 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jgzw\" (UniqueName: \"kubernetes.io/projected/ac902ebe-e400-491e-b04f-1791e8a711e3-kube-api-access-9jgzw\") pod \"cert-manager-webhook-6486ffd997-mm5q6\" (UID: \"ac902ebe-e400-491e-b04f-1791e8a711e3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:45.242876 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.242855 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac902ebe-e400-491e-b04f-1791e8a711e3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mm5q6\" (UID: \"ac902ebe-e400-491e-b04f-1791e8a711e3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:45.242927 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.242917 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.343955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.343877 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac902ebe-e400-491e-b04f-1791e8a711e3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mm5q6\" (UID: \"ac902ebe-e400-491e-b04f-1791e8a711e3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:45.343955 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.343925 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.344194 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.343969 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jddft\" (UniqueName: \"kubernetes.io/projected/de99ab8c-8910-4235-bbe5-eb71b825d69e-kube-api-access-jddft\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.344194 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.344023 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.344194 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.344076 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jgzw\" (UniqueName: \"kubernetes.io/projected/ac902ebe-e400-491e-b04f-1791e8a711e3-kube-api-access-9jgzw\") pod \"cert-manager-webhook-6486ffd997-mm5q6\" (UID: \"ac902ebe-e400-491e-b04f-1791e8a711e3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:45.344502 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.344479 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.344538 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.344506 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.356919 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.356899 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ac902ebe-e400-491e-b04f-1791e8a711e3-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mm5q6\" (UID: \"ac902ebe-e400-491e-b04f-1791e8a711e3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:45.357048 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.356985 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jgzw\" (UniqueName: \"kubernetes.io/projected/ac902ebe-e400-491e-b04f-1791e8a711e3-kube-api-access-9jgzw\") pod \"cert-manager-webhook-6486ffd997-mm5q6\" (UID: \"ac902ebe-e400-491e-b04f-1791e8a711e3\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:45.358450 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.358430 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jddft\" (UniqueName: \"kubernetes.io/projected/de99ab8c-8910-4235-bbe5-eb71b825d69e-kube-api-access-jddft\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.444982 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.444958 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:45.482265 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.482233 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:45.592783 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.592744 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mm5q6"] Jul 16 21:54:45.595410 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:54:45.595344 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac902ebe_e400_491e_b04f_1791e8a711e3.slice/crio-523c1559e92472720dc10f30a8bebcc687a5eb8458674176ff5669c2bded4749 WatchSource:0}: Error finding container 523c1559e92472720dc10f30a8bebcc687a5eb8458674176ff5669c2bded4749: Status 404 returned error can't find the container with id 523c1559e92472720dc10f30a8bebcc687a5eb8458674176ff5669c2bded4749 Jul 16 21:54:45.629325 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:45.629263 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr"] Jul 16 21:54:45.631465 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:54:45.631438 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde99ab8c_8910_4235_bbe5_eb71b825d69e.slice/crio-24af0ae1a7ecd8e5aa778ea1d20a3db87c4b19ecbe244cfa4da451cd03ebd3a2 WatchSource:0}: Error finding container 24af0ae1a7ecd8e5aa778ea1d20a3db87c4b19ecbe244cfa4da451cd03ebd3a2: Status 404 returned error can't find the container with id 24af0ae1a7ecd8e5aa778ea1d20a3db87c4b19ecbe244cfa4da451cd03ebd3a2 Jul 16 21:54:46.077814 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:46.077780 2563 generic.go:356] "Generic (PLEG): container finished" podID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerID="a31cb80b87de135b0c018cb7662783d230c2fd4f33cdead318f70060a2a6acb4" exitCode=0 Jul 16 21:54:46.077996 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:46.077870 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" event={"ID":"de99ab8c-8910-4235-bbe5-eb71b825d69e","Type":"ContainerDied","Data":"a31cb80b87de135b0c018cb7662783d230c2fd4f33cdead318f70060a2a6acb4"} Jul 16 21:54:46.077996 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:46.077926 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" event={"ID":"de99ab8c-8910-4235-bbe5-eb71b825d69e","Type":"ContainerStarted","Data":"24af0ae1a7ecd8e5aa778ea1d20a3db87c4b19ecbe244cfa4da451cd03ebd3a2"} Jul 16 21:54:46.079339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:46.079237 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" event={"ID":"ac902ebe-e400-491e-b04f-1791e8a711e3","Type":"ContainerStarted","Data":"523c1559e92472720dc10f30a8bebcc687a5eb8458674176ff5669c2bded4749"} Jul 16 21:54:48.215697 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.215660 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqszb"] Jul 16 21:54:48.218054 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.218032 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" Jul 16 21:54:48.221428 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.221405 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-rbpff\"" Jul 16 21:54:48.228297 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.228272 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqszb"] Jul 16 21:54:48.374066 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.374024 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b5b875e-6a82-46fb-8284-e2eb3471ac2e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqszb\" (UID: \"8b5b875e-6a82-46fb-8284-e2eb3471ac2e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" Jul 16 21:54:48.374241 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.374086 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4hmn\" (UniqueName: \"kubernetes.io/projected/8b5b875e-6a82-46fb-8284-e2eb3471ac2e-kube-api-access-j4hmn\") pod \"cert-manager-cainjector-559cdd8c87-rqszb\" (UID: \"8b5b875e-6a82-46fb-8284-e2eb3471ac2e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" Jul 16 21:54:48.475251 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.475163 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b5b875e-6a82-46fb-8284-e2eb3471ac2e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqszb\" (UID: \"8b5b875e-6a82-46fb-8284-e2eb3471ac2e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" Jul 16 21:54:48.475251 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.475216 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4hmn\" (UniqueName: \"kubernetes.io/projected/8b5b875e-6a82-46fb-8284-e2eb3471ac2e-kube-api-access-j4hmn\") pod \"cert-manager-cainjector-559cdd8c87-rqszb\" (UID: \"8b5b875e-6a82-46fb-8284-e2eb3471ac2e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" Jul 16 21:54:48.488379 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.488355 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8b5b875e-6a82-46fb-8284-e2eb3471ac2e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rqszb\" (UID: \"8b5b875e-6a82-46fb-8284-e2eb3471ac2e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" Jul 16 21:54:48.488557 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.488532 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4hmn\" (UniqueName: \"kubernetes.io/projected/8b5b875e-6a82-46fb-8284-e2eb3471ac2e-kube-api-access-j4hmn\") pod \"cert-manager-cainjector-559cdd8c87-rqszb\" (UID: \"8b5b875e-6a82-46fb-8284-e2eb3471ac2e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" Jul 16 21:54:48.530103 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.530067 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" Jul 16 21:54:48.879875 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:48.879853 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rqszb"] Jul 16 21:54:48.922942 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:54:48.922914 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b5b875e_6a82_46fb_8284_e2eb3471ac2e.slice/crio-42c028f1d93d300c453fd30eddcd8c7d726ac8aaacf25e2abcd35f76093598c2 WatchSource:0}: Error finding container 42c028f1d93d300c453fd30eddcd8c7d726ac8aaacf25e2abcd35f76093598c2: Status 404 returned error can't find the container with id 42c028f1d93d300c453fd30eddcd8c7d726ac8aaacf25e2abcd35f76093598c2 Jul 16 21:54:49.098851 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:49.098736 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" event={"ID":"8b5b875e-6a82-46fb-8284-e2eb3471ac2e","Type":"ContainerStarted","Data":"141ef4a695e27cfa7c36d207192d2b5f6c85db8fe368bd55f50d8a3b40b90996"} Jul 16 21:54:49.098851 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:49.098798 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" event={"ID":"8b5b875e-6a82-46fb-8284-e2eb3471ac2e","Type":"ContainerStarted","Data":"42c028f1d93d300c453fd30eddcd8c7d726ac8aaacf25e2abcd35f76093598c2"} Jul 16 21:54:49.100439 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:49.100413 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" event={"ID":"ac902ebe-e400-491e-b04f-1791e8a711e3","Type":"ContainerStarted","Data":"cdbc3861f27b15d5c4da2c69714e045bf81cc1c68dfad6a53aec5f04a5f77d4f"} Jul 16 21:54:49.100582 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:49.100469 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:49.102385 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:49.102354 2563 generic.go:356] "Generic (PLEG): container finished" podID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerID="c8fc518cc2b4c1dc656932ff89e6dee780058eb4b235e5af82830d3c3dd91781" exitCode=0 Jul 16 21:54:49.102503 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:49.102398 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" event={"ID":"de99ab8c-8910-4235-bbe5-eb71b825d69e","Type":"ContainerDied","Data":"c8fc518cc2b4c1dc656932ff89e6dee780058eb4b235e5af82830d3c3dd91781"} Jul 16 21:54:49.119100 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:49.119056 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rqszb" podStartSLOduration=1.11904197 podStartE2EDuration="1.11904197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:49.116033785 +0000 UTC m=+484.449112818" watchObservedRunningTime="2026-07-16 21:54:49.11904197 +0000 UTC m=+484.452120999" Jul 16 21:54:49.158391 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:49.158339 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" podStartSLOduration=1.012224814 podStartE2EDuration="4.158317237s" totalImagesPullingTime="3.146092423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:49.157101145 +0000 UTC m=+484.490180176" watchObservedRunningTime="2026-07-16 21:54:49.158317237 +0000 UTC m=+484.491396260" Jul 16 21:54:50.109391 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:50.109350 2563 generic.go:356] "Generic (PLEG): container finished" podID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerID="d9a7881f365f275847539b706dc64f5200015acdce8a7b3f73b7dce64669cffa" exitCode=0 Jul 16 21:54:50.109768 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:50.109437 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" event={"ID":"de99ab8c-8910-4235-bbe5-eb71b825d69e","Type":"ContainerDied","Data":"d9a7881f365f275847539b706dc64f5200015acdce8a7b3f73b7dce64669cffa"} Jul 16 21:54:51.246695 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.246670 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:51.398034 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.397943 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-util\") pod \"de99ab8c-8910-4235-bbe5-eb71b825d69e\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " Jul 16 21:54:51.398034 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.398000 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jddft\" (UniqueName: \"kubernetes.io/projected/de99ab8c-8910-4235-bbe5-eb71b825d69e-kube-api-access-jddft\") pod \"de99ab8c-8910-4235-bbe5-eb71b825d69e\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " Jul 16 21:54:51.398285 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.398112 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-bundle\") pod \"de99ab8c-8910-4235-bbe5-eb71b825d69e\" (UID: \"de99ab8c-8910-4235-bbe5-eb71b825d69e\") " Jul 16 21:54:51.398548 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.398518 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-bundle" (OuterVolumeSpecName: "bundle") pod "de99ab8c-8910-4235-bbe5-eb71b825d69e" (UID: "de99ab8c-8910-4235-bbe5-eb71b825d69e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:54:51.400186 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.400121 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de99ab8c-8910-4235-bbe5-eb71b825d69e-kube-api-access-jddft" (OuterVolumeSpecName: "kube-api-access-jddft") pod "de99ab8c-8910-4235-bbe5-eb71b825d69e" (UID: "de99ab8c-8910-4235-bbe5-eb71b825d69e"). InnerVolumeSpecName "kube-api-access-jddft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:54:51.402993 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.402973 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-util" (OuterVolumeSpecName: "util") pod "de99ab8c-8910-4235-bbe5-eb71b825d69e" (UID: "de99ab8c-8910-4235-bbe5-eb71b825d69e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:54:51.498713 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.498687 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:51.498713 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.498710 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de99ab8c-8910-4235-bbe5-eb71b825d69e-util\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:51.498836 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:51.498720 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jddft\" (UniqueName: \"kubernetes.io/projected/de99ab8c-8910-4235-bbe5-eb71b825d69e-kube-api-access-jddft\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:54:52.120036 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:52.120003 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" event={"ID":"de99ab8c-8910-4235-bbe5-eb71b825d69e","Type":"ContainerDied","Data":"24af0ae1a7ecd8e5aa778ea1d20a3db87c4b19ecbe244cfa4da451cd03ebd3a2"} Jul 16 21:54:52.120036 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:52.120027 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzmxbr" Jul 16 21:54:52.120249 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:52.120050 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24af0ae1a7ecd8e5aa778ea1d20a3db87c4b19ecbe244cfa4da451cd03ebd3a2" Jul 16 21:54:55.111934 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:55.111903 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mm5q6" Jul 16 21:54:59.351157 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.351114 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww"] Jul 16 21:54:59.351520 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.351464 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerName="util" Jul 16 21:54:59.351520 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.351476 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerName="util" Jul 16 21:54:59.351520 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.351496 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerName="pull" Jul 16 21:54:59.351520 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.351501 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerName="pull" Jul 16 21:54:59.351520 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.351507 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerName="extract" Jul 16 21:54:59.351520 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.351514 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerName="extract" Jul 16 21:54:59.351721 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.351567 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="de99ab8c-8910-4235-bbe5-eb71b825d69e" containerName="extract" Jul 16 21:54:59.355050 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.355033 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" Jul 16 21:54:59.358328 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.358304 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 21:54:59.358455 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.358345 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 21:54:59.359866 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.359842 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kg4fv\"" Jul 16 21:54:59.364186 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.364162 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww"] Jul 16 21:54:59.365837 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.365816 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx2wm\" (UniqueName: \"kubernetes.io/projected/ebda4572-5df2-42cf-a5ce-4c4521edaf49-kube-api-access-vx2wm\") pod \"openshift-lws-operator-fd8ccff4c-nf8ww\" (UID: \"ebda4572-5df2-42cf-a5ce-4c4521edaf49\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" Jul 16 21:54:59.365941 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.365847 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebda4572-5df2-42cf-a5ce-4c4521edaf49-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nf8ww\" (UID: \"ebda4572-5df2-42cf-a5ce-4c4521edaf49\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" Jul 16 21:54:59.466529 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.466499 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx2wm\" (UniqueName: \"kubernetes.io/projected/ebda4572-5df2-42cf-a5ce-4c4521edaf49-kube-api-access-vx2wm\") pod \"openshift-lws-operator-fd8ccff4c-nf8ww\" (UID: \"ebda4572-5df2-42cf-a5ce-4c4521edaf49\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" Jul 16 21:54:59.466652 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.466538 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebda4572-5df2-42cf-a5ce-4c4521edaf49-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nf8ww\" (UID: \"ebda4572-5df2-42cf-a5ce-4c4521edaf49\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" Jul 16 21:54:59.466880 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.466864 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebda4572-5df2-42cf-a5ce-4c4521edaf49-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nf8ww\" (UID: \"ebda4572-5df2-42cf-a5ce-4c4521edaf49\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" Jul 16 21:54:59.476009 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.475982 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx2wm\" (UniqueName: \"kubernetes.io/projected/ebda4572-5df2-42cf-a5ce-4c4521edaf49-kube-api-access-vx2wm\") pod \"openshift-lws-operator-fd8ccff4c-nf8ww\" (UID: \"ebda4572-5df2-42cf-a5ce-4c4521edaf49\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" Jul 16 21:54:59.665193 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.665112 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" Jul 16 21:54:59.795864 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:54:59.795836 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww"] Jul 16 21:54:59.797747 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:54:59.797720 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebda4572_5df2_42cf_a5ce_4c4521edaf49.slice/crio-c61e1e38f04223b68651b4c9083241394599411ef644dad92c011a6e72866d52 WatchSource:0}: Error finding container c61e1e38f04223b68651b4c9083241394599411ef644dad92c011a6e72866d52: Status 404 returned error can't find the container with id c61e1e38f04223b68651b4c9083241394599411ef644dad92c011a6e72866d52 Jul 16 21:55:00.163181 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:00.163123 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" event={"ID":"ebda4572-5df2-42cf-a5ce-4c4521edaf49","Type":"ContainerStarted","Data":"c61e1e38f04223b68651b4c9083241394599411ef644dad92c011a6e72866d52"} Jul 16 21:55:02.174383 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:02.174283 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" event={"ID":"ebda4572-5df2-42cf-a5ce-4c4521edaf49","Type":"ContainerStarted","Data":"acbd2e68af31c2e0c0f2376e9f99156c339653dab806cf8654ccc3096440500e"} Jul 16 21:55:02.192432 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:02.192388 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nf8ww" podStartSLOduration=1.136502113 podStartE2EDuration="3.192372792s" totalImagesPullingTime="2.055870679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:54:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:02.191484036 +0000 UTC m=+497.524563067" watchObservedRunningTime="2026-07-16 21:55:02.192372792 +0000 UTC m=+497.525451821" Jul 16 21:55:04.950442 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:04.950402 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh"] Jul 16 21:55:04.967228 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:04.967203 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh"] Jul 16 21:55:04.967375 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:04.967299 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:04.970168 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:04.970131 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 21:55:04.971461 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:04.971436 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tdcn5\"" Jul 16 21:55:04.971461 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:04.971451 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 21:55:05.021710 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.021685 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.021826 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.021728 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96h7t\" (UniqueName: \"kubernetes.io/projected/fb968145-3726-4a38-b460-74b8004a919b-kube-api-access-96h7t\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.021826 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.021797 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.122204 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.122176 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.122323 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.122234 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.122323 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.122274 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96h7t\" (UniqueName: \"kubernetes.io/projected/fb968145-3726-4a38-b460-74b8004a919b-kube-api-access-96h7t\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.122552 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.122525 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.122652 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.122581 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.135500 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.135474 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96h7t\" (UniqueName: \"kubernetes.io/projected/fb968145-3726-4a38-b460-74b8004a919b-kube-api-access-96h7t\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.276993 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.276928 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:05.418539 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:05.418518 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh"] Jul 16 21:55:05.420518 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:55:05.420492 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb968145_3726_4a38_b460_74b8004a919b.slice/crio-b8fcfd15dd5a19fda6a720e3a21b05b509d91691b0e1cd256b9ad3dac1f036ff WatchSource:0}: Error finding container b8fcfd15dd5a19fda6a720e3a21b05b509d91691b0e1cd256b9ad3dac1f036ff: Status 404 returned error can't find the container with id b8fcfd15dd5a19fda6a720e3a21b05b509d91691b0e1cd256b9ad3dac1f036ff Jul 16 21:55:06.194387 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:06.194355 2563 generic.go:356] "Generic (PLEG): container finished" podID="fb968145-3726-4a38-b460-74b8004a919b" containerID="6886832be11d5c0b3e934e81ac6fba457594c622a2c1a00dcb8f0ce4417f4d86" exitCode=0 Jul 16 21:55:06.194742 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:06.194450 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" event={"ID":"fb968145-3726-4a38-b460-74b8004a919b","Type":"ContainerDied","Data":"6886832be11d5c0b3e934e81ac6fba457594c622a2c1a00dcb8f0ce4417f4d86"} Jul 16 21:55:06.194742 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:06.194518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" event={"ID":"fb968145-3726-4a38-b460-74b8004a919b","Type":"ContainerStarted","Data":"b8fcfd15dd5a19fda6a720e3a21b05b509d91691b0e1cd256b9ad3dac1f036ff"} Jul 16 21:55:07.200294 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:07.200263 2563 generic.go:356] "Generic (PLEG): container finished" podID="fb968145-3726-4a38-b460-74b8004a919b" containerID="fc9f671f49d26683b703f279ac76aaa58b0dcc375a1eb0515847c5b61c373301" exitCode=0 Jul 16 21:55:07.200715 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:07.200337 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" event={"ID":"fb968145-3726-4a38-b460-74b8004a919b","Type":"ContainerDied","Data":"fc9f671f49d26683b703f279ac76aaa58b0dcc375a1eb0515847c5b61c373301"} Jul 16 21:55:08.212613 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:08.212579 2563 generic.go:356] "Generic (PLEG): container finished" podID="fb968145-3726-4a38-b460-74b8004a919b" containerID="f1326d87b91cd3c28daf63a19187707f7bf0fe0392bb57379414dd733257c8f3" exitCode=0 Jul 16 21:55:08.213004 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:08.212850 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" event={"ID":"fb968145-3726-4a38-b460-74b8004a919b","Type":"ContainerDied","Data":"f1326d87b91cd3c28daf63a19187707f7bf0fe0392bb57379414dd733257c8f3"} Jul 16 21:55:09.348532 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.348500 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:09.455108 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.455026 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-util\") pod \"fb968145-3726-4a38-b460-74b8004a919b\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " Jul 16 21:55:09.455270 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.455215 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96h7t\" (UniqueName: \"kubernetes.io/projected/fb968145-3726-4a38-b460-74b8004a919b-kube-api-access-96h7t\") pod \"fb968145-3726-4a38-b460-74b8004a919b\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " Jul 16 21:55:09.455270 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.455239 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-bundle\") pod \"fb968145-3726-4a38-b460-74b8004a919b\" (UID: \"fb968145-3726-4a38-b460-74b8004a919b\") " Jul 16 21:55:09.455963 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.455937 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-bundle" (OuterVolumeSpecName: "bundle") pod "fb968145-3726-4a38-b460-74b8004a919b" (UID: "fb968145-3726-4a38-b460-74b8004a919b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:55:09.457339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.457313 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb968145-3726-4a38-b460-74b8004a919b-kube-api-access-96h7t" (OuterVolumeSpecName: "kube-api-access-96h7t") pod "fb968145-3726-4a38-b460-74b8004a919b" (UID: "fb968145-3726-4a38-b460-74b8004a919b"). InnerVolumeSpecName "kube-api-access-96h7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:55:09.460477 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.460446 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-util" (OuterVolumeSpecName: "util") pod "fb968145-3726-4a38-b460-74b8004a919b" (UID: "fb968145-3726-4a38-b460-74b8004a919b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:55:09.556865 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.556806 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96h7t\" (UniqueName: \"kubernetes.io/projected/fb968145-3726-4a38-b460-74b8004a919b-kube-api-access-96h7t\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:55:09.556865 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.556829 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:55:09.556865 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:09.556838 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb968145-3726-4a38-b460-74b8004a919b-util\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:55:10.224071 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:10.224040 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" Jul 16 21:55:10.224071 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:10.224054 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afql8wh" event={"ID":"fb968145-3726-4a38-b460-74b8004a919b","Type":"ContainerDied","Data":"b8fcfd15dd5a19fda6a720e3a21b05b509d91691b0e1cd256b9ad3dac1f036ff"} Jul 16 21:55:10.224346 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:10.224094 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8fcfd15dd5a19fda6a720e3a21b05b509d91691b0e1cd256b9ad3dac1f036ff" Jul 16 21:55:21.688479 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.688446 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df6666d-t64df"] Jul 16 21:55:21.688852 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.688778 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb968145-3726-4a38-b460-74b8004a919b" containerName="pull" Jul 16 21:55:21.688852 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.688789 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb968145-3726-4a38-b460-74b8004a919b" containerName="pull" Jul 16 21:55:21.688852 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.688806 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb968145-3726-4a38-b460-74b8004a919b" containerName="util" Jul 16 21:55:21.688852 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.688811 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb968145-3726-4a38-b460-74b8004a919b" containerName="util" Jul 16 21:55:21.688852 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.688827 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb968145-3726-4a38-b460-74b8004a919b" containerName="extract" Jul 16 21:55:21.688852 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.688833 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb968145-3726-4a38-b460-74b8004a919b" containerName="extract" Jul 16 21:55:21.689037 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.688885 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb968145-3726-4a38-b460-74b8004a919b" containerName="extract" Jul 16 21:55:21.694339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.694319 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.701102 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.701080 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 21:55:21.701215 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.701093 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q5qbl\"" Jul 16 21:55:21.701429 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.701413 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 21:55:21.702256 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.702239 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 21:55:21.702530 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.702517 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 21:55:21.725197 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.725172 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df6666d-t64df"] Jul 16 21:55:21.762387 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.762365 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12738948-2307-42cd-a500-893d2835caa9-tmp\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.762484 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.762391 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12738948-2307-42cd-a500-893d2835caa9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.762484 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.762412 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6w2c\" (UniqueName: \"kubernetes.io/projected/12738948-2307-42cd-a500-893d2835caa9-kube-api-access-j6w2c\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.762556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.762509 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12738948-2307-42cd-a500-893d2835caa9-manifests\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.762590 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.762568 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12738948-2307-42cd-a500-893d2835caa9-webhook-cert\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.863342 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.863302 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12738948-2307-42cd-a500-893d2835caa9-manifests\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.863519 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.863359 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12738948-2307-42cd-a500-893d2835caa9-webhook-cert\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.863519 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.863394 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12738948-2307-42cd-a500-893d2835caa9-tmp\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.863519 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.863413 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12738948-2307-42cd-a500-893d2835caa9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.863519 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.863434 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6w2c\" (UniqueName: \"kubernetes.io/projected/12738948-2307-42cd-a500-893d2835caa9-kube-api-access-j6w2c\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.864263 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.864231 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12738948-2307-42cd-a500-893d2835caa9-manifests\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.866631 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.866600 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12738948-2307-42cd-a500-893d2835caa9-tmp\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.871499 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.866946 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12738948-2307-42cd-a500-893d2835caa9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.871499 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.867054 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12738948-2307-42cd-a500-893d2835caa9-webhook-cert\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:21.876870 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:21.876839 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6w2c\" (UniqueName: \"kubernetes.io/projected/12738948-2307-42cd-a500-893d2835caa9-kube-api-access-j6w2c\") pod \"opendatahub-operator-controller-manager-df6666d-t64df\" (UID: \"12738948-2307-42cd-a500-893d2835caa9\") " pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:22.005284 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:22.005210 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:22.182418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:22.182387 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df6666d-t64df"] Jul 16 21:55:22.183521 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:55:22.183492 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12738948_2307_42cd_a500_893d2835caa9.slice/crio-6c442589cd2fdcea7a932e66541fb008494c72a17564c8d976e064824a5544a8 WatchSource:0}: Error finding container 6c442589cd2fdcea7a932e66541fb008494c72a17564c8d976e064824a5544a8: Status 404 returned error can't find the container with id 6c442589cd2fdcea7a932e66541fb008494c72a17564c8d976e064824a5544a8 Jul 16 21:55:22.279351 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:22.279279 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" event={"ID":"12738948-2307-42cd-a500-893d2835caa9","Type":"ContainerStarted","Data":"6c442589cd2fdcea7a932e66541fb008494c72a17564c8d976e064824a5544a8"} Jul 16 21:55:25.297510 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:25.297478 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" event={"ID":"12738948-2307-42cd-a500-893d2835caa9","Type":"ContainerStarted","Data":"ce04e76eaf6a9f85d74371fc4c95328d337c0128215c019d0ed7c60105b70c69"} Jul 16 21:55:26.304369 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:26.304339 2563 generic.go:356] "Generic (PLEG): container finished" podID="12738948-2307-42cd-a500-893d2835caa9" containerID="ce04e76eaf6a9f85d74371fc4c95328d337c0128215c019d0ed7c60105b70c69" exitCode=0 Jul 16 21:55:26.304679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:26.304428 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" event={"ID":"12738948-2307-42cd-a500-893d2835caa9","Type":"ContainerDied","Data":"ce04e76eaf6a9f85d74371fc4c95328d337c0128215c019d0ed7c60105b70c69"} Jul 16 21:55:27.311471 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:27.311434 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" event={"ID":"12738948-2307-42cd-a500-893d2835caa9","Type":"ContainerStarted","Data":"b89a528b5b200e632292fe5fe336141c98679ab7943b2b4df19f51cb6e187b0b"} Jul 16 21:55:27.311853 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:27.311514 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:38.318303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:38.318228 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" Jul 16 21:55:38.372623 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:38.372576 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-df6666d-t64df" podStartSLOduration=12.931849614 podStartE2EDuration="17.372560888s" totalImagesPullingTime="3.359583276s" totalInitContainerRuntime="1.081127998s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:27.472090438 +0000 UTC m=+522.805169467" watchObservedRunningTime="2026-07-16 21:55:38.372560888 +0000 UTC m=+533.705639918" Jul 16 21:55:43.399498 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.399460 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xhrkd"] Jul 16 21:55:43.401965 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.401943 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:43.447434 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.447409 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 16 21:55:43.447871 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.447847 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-g4j4b\"" Jul 16 21:55:43.452221 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.452192 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert\") pod \"odh-model-controller-7998c67dc4-xhrkd\" (UID: \"6edbb00f-c805-4dff-8e45-7497cd417530\") " pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:43.452335 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.452261 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfqjm\" (UniqueName: \"kubernetes.io/projected/6edbb00f-c805-4dff-8e45-7497cd417530-kube-api-access-bfqjm\") pod \"odh-model-controller-7998c67dc4-xhrkd\" (UID: \"6edbb00f-c805-4dff-8e45-7497cd417530\") " pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:43.512054 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.512029 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xhrkd"] Jul 16 21:55:43.552708 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.552677 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert\") pod \"odh-model-controller-7998c67dc4-xhrkd\" (UID: \"6edbb00f-c805-4dff-8e45-7497cd417530\") " pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:43.552816 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.552752 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfqjm\" (UniqueName: \"kubernetes.io/projected/6edbb00f-c805-4dff-8e45-7497cd417530-kube-api-access-bfqjm\") pod \"odh-model-controller-7998c67dc4-xhrkd\" (UID: \"6edbb00f-c805-4dff-8e45-7497cd417530\") " pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:43.552859 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:43.552807 2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 16 21:55:43.552893 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:43.552864 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert podName:6edbb00f-c805-4dff-8e45-7497cd417530 nodeName:}" failed. No retries permitted until 2026-07-16 21:55:44.052849684 +0000 UTC m=+539.385928692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert") pod "odh-model-controller-7998c67dc4-xhrkd" (UID: "6edbb00f-c805-4dff-8e45-7497cd417530") : secret "odh-model-controller-webhook-cert" not found Jul 16 21:55:43.618868 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:43.618834 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfqjm\" (UniqueName: \"kubernetes.io/projected/6edbb00f-c805-4dff-8e45-7497cd417530-kube-api-access-bfqjm\") pod \"odh-model-controller-7998c67dc4-xhrkd\" (UID: \"6edbb00f-c805-4dff-8e45-7497cd417530\") " pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:44.057738 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:44.057706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert\") pod \"odh-model-controller-7998c67dc4-xhrkd\" (UID: \"6edbb00f-c805-4dff-8e45-7497cd417530\") " pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:44.057912 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:44.057854 2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 16 21:55:44.057956 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:44.057923 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert podName:6edbb00f-c805-4dff-8e45-7497cd417530 nodeName:}" failed. No retries permitted until 2026-07-16 21:55:45.057906438 +0000 UTC m=+540.390985447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert") pod "odh-model-controller-7998c67dc4-xhrkd" (UID: "6edbb00f-c805-4dff-8e45-7497cd417530") : secret "odh-model-controller-webhook-cert" not found Jul 16 21:55:45.065816 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:45.065785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert\") pod \"odh-model-controller-7998c67dc4-xhrkd\" (UID: \"6edbb00f-c805-4dff-8e45-7497cd417530\") " pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:45.068202 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:45.068179 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6edbb00f-c805-4dff-8e45-7497cd417530-cert\") pod \"odh-model-controller-7998c67dc4-xhrkd\" (UID: \"6edbb00f-c805-4dff-8e45-7497cd417530\") " pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:45.215287 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:45.215260 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-g4j4b\"" Jul 16 21:55:45.222681 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:45.222646 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:45.656090 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:55:45.656057 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6edbb00f_c805_4dff_8e45_7497cd417530.slice/crio-3756506f6ccaac3d93759a24ae899f8d48f42d74c1305040ea8e1922b9c41615 WatchSource:0}: Error finding container 3756506f6ccaac3d93759a24ae899f8d48f42d74c1305040ea8e1922b9c41615: Status 404 returned error can't find the container with id 3756506f6ccaac3d93759a24ae899f8d48f42d74c1305040ea8e1922b9c41615 Jul 16 21:55:45.659403 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:45.659379 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xhrkd"] Jul 16 21:55:46.403331 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:46.403294 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" event={"ID":"6edbb00f-c805-4dff-8e45-7497cd417530","Type":"ContainerStarted","Data":"3756506f6ccaac3d93759a24ae899f8d48f42d74c1305040ea8e1922b9c41615"} Jul 16 21:55:48.415359 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:48.415319 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" event={"ID":"6edbb00f-c805-4dff-8e45-7497cd417530","Type":"ContainerStarted","Data":"08e1f1005d58ec566282f2ea22c7dcc766d9039bc8fccde18e416083e7417d1c"} Jul 16 21:55:48.415738 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:48.415411 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:55:48.514447 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:48.514340 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" podStartSLOduration=2.965351269 podStartE2EDuration="5.514325432s" totalImagesPullingTime="2.548974163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:48.512601675 +0000 UTC m=+543.845680704" watchObservedRunningTime="2026-07-16 21:55:48.514325432 +0000 UTC m=+543.847404462" Jul 16 21:55:49.559984 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.559954 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq"] Jul 16 21:55:49.562344 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.562327 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.566337 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:49.566303 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istio-kubeconfig\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" type="*v1.Secret" Jul 16 21:55:49.567164 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:49.567126 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-gw-ca-root-cert\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap" Jul 16 21:55:49.567549 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:49.567526 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-openshift-gateway-dockercfg-vz5h5\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vz5h5\"" type="*v1.Secret" Jul 16 21:55:49.567632 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:49.567593 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-tls\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" type="*v1.Secret" Jul 16 21:55:49.567804 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:49.567778 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"istiod-openshift-gateway-7574459656-2pdkq\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" podUID="b4de1cee-9308-4757-9033-021dddf0e53a" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.567998 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:49.567972 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cacerts\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"cacerts\"" type="*v1.Secret" Jul 16 21:55:49.608558 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.608534 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vtg9\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-kube-api-access-5vtg9\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.608675 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.608619 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4de1cee-9308-4757-9033-021dddf0e53a-local-certs\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.608675 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.608658 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.608803 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.608729 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-istio-token\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.608803 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.608771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.608907 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.608830 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.608907 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.608855 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.647741 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.647714 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq"] Jul 16 21:55:49.709649 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.709617 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.709649 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.709650 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.709841 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.709669 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vtg9\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-kube-api-access-5vtg9\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.709841 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.709714 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4de1cee-9308-4757-9033-021dddf0e53a-local-certs\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.709841 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.709798 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.709976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.709852 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-istio-token\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.709976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.709883 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.711882 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.711855 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4de1cee-9308-4757-9033-021dddf0e53a-local-certs\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.743663 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.743637 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vtg9\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-kube-api-access-5vtg9\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:49.747601 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:49.747575 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-istio-token\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:50.566360 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:50.566329 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 21:55:50.572484 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:50.572459 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:50.581654 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:50.581632 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 21:55:50.590456 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:50.590433 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:50.710817 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:50.710787 2563 secret.go:189] Couldn't get secret openshift-ingress/cacerts: failed to sync secret cache: timed out waiting for the condition Jul 16 21:55:50.710953 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:50.710856 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts podName:b4de1cee-9308-4757-9033-021dddf0e53a nodeName:}" failed. No retries permitted until 2026-07-16 21:55:51.210837263 +0000 UTC m=+546.543916274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cacerts" (UniqueName: "kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts") pod "istiod-openshift-gateway-7574459656-2pdkq" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a") : failed to sync secret cache: timed out waiting for the condition Jul 16 21:55:50.710953 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:50.710787 2563 secret.go:189] Couldn't get secret openshift-ingress/istio-kubeconfig: failed to sync secret cache: timed out waiting for the condition Jul 16 21:55:50.710953 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:55:50.710945 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig podName:b4de1cee-9308-4757-9033-021dddf0e53a nodeName:}" failed. No retries permitted until 2026-07-16 21:55:51.210928117 +0000 UTC m=+546.544007141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-kubeconfig" (UniqueName: "kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig") pod "istiod-openshift-gateway-7574459656-2pdkq" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a") : failed to sync secret cache: timed out waiting for the condition Jul 16 21:55:51.042046 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.042014 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vz5h5\"" Jul 16 21:55:51.081763 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.081738 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 21:55:51.197255 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.197225 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 21:55:51.223151 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.223123 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:51.223264 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.223224 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:51.225424 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.225399 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:51.225521 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.225471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-2pdkq\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:51.372094 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.372063 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:51.672458 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:55:51.672424 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4de1cee_9308_4757_9033_021dddf0e53a.slice/crio-5952c4ef15b022d786bc3dac6e751a6552f557843bc9ac493424b3dff3f891f7 WatchSource:0}: Error finding container 5952c4ef15b022d786bc3dac6e751a6552f557843bc9ac493424b3dff3f891f7: Status 404 returned error can't find the container with id 5952c4ef15b022d786bc3dac6e751a6552f557843bc9ac493424b3dff3f891f7 Jul 16 21:55:51.672755 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:51.672674 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq"] Jul 16 21:55:52.465822 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:52.465777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" event={"ID":"b4de1cee-9308-4757-9033-021dddf0e53a","Type":"ContainerStarted","Data":"5952c4ef15b022d786bc3dac6e751a6552f557843bc9ac493424b3dff3f891f7"} Jul 16 21:55:54.092850 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.092800 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 21:55:54.093106 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.092880 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 21:55:54.294974 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.294938 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt"] Jul 16 21:55:54.298541 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.298518 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.310781 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.310762 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 21:55:54.310891 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.310777 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kt5xj\"" Jul 16 21:55:54.310891 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.310777 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 21:55:54.310891 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.310762 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 21:55:54.336210 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.336188 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt"] Jul 16 21:55:54.355708 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.355688 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/577542db-ff6d-4a89-a68d-5031a26fbbb4-cert\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.355846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.355725 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/577542db-ff6d-4a89-a68d-5031a26fbbb4-metrics-cert\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.355846 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.355743 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmsj7\" (UniqueName: \"kubernetes.io/projected/577542db-ff6d-4a89-a68d-5031a26fbbb4-kube-api-access-nmsj7\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.355956 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.355860 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/577542db-ff6d-4a89-a68d-5031a26fbbb4-manager-config\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.457430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.457398 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/577542db-ff6d-4a89-a68d-5031a26fbbb4-metrics-cert\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.457583 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.457442 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmsj7\" (UniqueName: \"kubernetes.io/projected/577542db-ff6d-4a89-a68d-5031a26fbbb4-kube-api-access-nmsj7\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.457583 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.457518 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/577542db-ff6d-4a89-a68d-5031a26fbbb4-manager-config\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.457673 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.457602 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/577542db-ff6d-4a89-a68d-5031a26fbbb4-cert\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.458329 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.458300 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/577542db-ff6d-4a89-a68d-5031a26fbbb4-manager-config\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.459883 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.459856 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/577542db-ff6d-4a89-a68d-5031a26fbbb4-metrics-cert\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.459883 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.459869 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/577542db-ff6d-4a89-a68d-5031a26fbbb4-cert\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.478918 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.478888 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" event={"ID":"b4de1cee-9308-4757-9033-021dddf0e53a","Type":"ContainerStarted","Data":"5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397"} Jul 16 21:55:54.479009 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.478949 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:54.496314 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.496285 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmsj7\" (UniqueName: \"kubernetes.io/projected/577542db-ff6d-4a89-a68d-5031a26fbbb4-kube-api-access-nmsj7\") pod \"lws-controller-manager-6c6c96995f-mhmwt\" (UID: \"577542db-ff6d-4a89-a68d-5031a26fbbb4\") " pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.525421 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.525376 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" podStartSLOduration=3.107347275 podStartE2EDuration="5.525361006s" totalImagesPullingTime="2.418013731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:54.524453319 +0000 UTC m=+549.857532360" watchObservedRunningTime="2026-07-16 21:55:54.525361006 +0000 UTC m=+549.858440035" Jul 16 21:55:54.610049 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.609966 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:54.801650 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:54.801624 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt"] Jul 16 21:55:54.803118 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:55:54.803092 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod577542db_ff6d_4a89_a68d_5031a26fbbb4.slice/crio-e58585f84c6383dd6d4a2a20a98b029055b23b23bdff9718d5e9f81bcfd3bb31 WatchSource:0}: Error finding container e58585f84c6383dd6d4a2a20a98b029055b23b23bdff9718d5e9f81bcfd3bb31: Status 404 returned error can't find the container with id e58585f84c6383dd6d4a2a20a98b029055b23b23bdff9718d5e9f81bcfd3bb31 Jul 16 21:55:55.489100 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:55.489068 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" event={"ID":"577542db-ff6d-4a89-a68d-5031a26fbbb4","Type":"ContainerStarted","Data":"e58585f84c6383dd6d4a2a20a98b029055b23b23bdff9718d5e9f81bcfd3bb31"} Jul 16 21:55:55.491187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:55.491138 2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-2pdkq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 16 21:55:55.491314 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:55.491212 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" podUID="b4de1cee-9308-4757-9033-021dddf0e53a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 21:55:57.500827 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:57.500787 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" event={"ID":"577542db-ff6d-4a89-a68d-5031a26fbbb4","Type":"ContainerStarted","Data":"1ebfe8c3c5c8139502fd544815053831a25c9138f2222ead8c838f964765cc0d"} Jul 16 21:55:57.501252 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:57.500927 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:55:57.666462 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:57.666408 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" podStartSLOduration=2.00302819 podStartE2EDuration="3.666390479s" totalImagesPullingTime="1.663362289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:55:57.66545995 +0000 UTC m=+552.998538979" watchObservedRunningTime="2026-07-16 21:55:57.666390479 +0000 UTC m=+552.999469508" Jul 16 21:55:58.490627 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:58.490596 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:55:59.421668 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:55:59.421638 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xhrkd" Jul 16 21:56:00.558734 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:00.558702 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq"] Jul 16 21:56:00.559199 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:00.558861 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" podUID="b4de1cee-9308-4757-9033-021dddf0e53a" containerName="discovery" containerID="cri-o://5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397" gracePeriod=30 Jul 16 21:56:00.908651 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:00.908627 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:56:01.014034 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.014006 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-ca-configmap\") pod \"b4de1cee-9308-4757-9033-021dddf0e53a\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " Jul 16 21:56:01.014214 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.014069 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-dns-cert\") pod \"b4de1cee-9308-4757-9033-021dddf0e53a\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " Jul 16 21:56:01.014214 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.014090 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts\") pod \"b4de1cee-9308-4757-9033-021dddf0e53a\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " Jul 16 21:56:01.014214 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.014134 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig\") pod \"b4de1cee-9308-4757-9033-021dddf0e53a\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " Jul 16 21:56:01.014214 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.014184 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-istio-token\") pod \"b4de1cee-9308-4757-9033-021dddf0e53a\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " Jul 16 21:56:01.014214 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.014206 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4de1cee-9308-4757-9033-021dddf0e53a-local-certs\") pod \"b4de1cee-9308-4757-9033-021dddf0e53a\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " Jul 16 21:56:01.014546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.014237 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vtg9\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-kube-api-access-5vtg9\") pod \"b4de1cee-9308-4757-9033-021dddf0e53a\" (UID: \"b4de1cee-9308-4757-9033-021dddf0e53a\") " Jul 16 21:56:01.014546 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.014512 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b4de1cee-9308-4757-9033-021dddf0e53a" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:56:01.016737 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.016700 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-kube-api-access-5vtg9" (OuterVolumeSpecName: "kube-api-access-5vtg9") pod "b4de1cee-9308-4757-9033-021dddf0e53a" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a"). InnerVolumeSpecName "kube-api-access-5vtg9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:56:01.016858 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.016775 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts" (OuterVolumeSpecName: "cacerts") pod "b4de1cee-9308-4757-9033-021dddf0e53a" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:56:01.016858 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.016807 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4de1cee-9308-4757-9033-021dddf0e53a-local-certs" (OuterVolumeSpecName: "local-certs") pod "b4de1cee-9308-4757-9033-021dddf0e53a" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:56:01.016858 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.016848 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b4de1cee-9308-4757-9033-021dddf0e53a" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:56:01.016970 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.016876 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4de1cee-9308-4757-9033-021dddf0e53a" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:56:01.016970 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.016894 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b4de1cee-9308-4757-9033-021dddf0e53a" (UID: "b4de1cee-9308-4757-9033-021dddf0e53a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 21:56:01.115857 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.115835 2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-dns-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:56:01.115857 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.115857 2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-cacerts\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:56:01.115997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.115867 2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4de1cee-9308-4757-9033-021dddf0e53a-istio-kubeconfig\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:56:01.115997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.115876 2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-istio-token\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:56:01.115997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.115884 2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4de1cee-9308-4757-9033-021dddf0e53a-local-certs\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:56:01.115997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.115893 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vtg9\" (UniqueName: \"kubernetes.io/projected/b4de1cee-9308-4757-9033-021dddf0e53a-kube-api-access-5vtg9\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:56:01.115997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.115902 2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b4de1cee-9308-4757-9033-021dddf0e53a-istio-csr-ca-configmap\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:56:01.521679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.521595 2563 generic.go:356] "Generic (PLEG): container finished" podID="b4de1cee-9308-4757-9033-021dddf0e53a" containerID="5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397" exitCode=0 Jul 16 21:56:01.521679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.521637 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" Jul 16 21:56:01.521915 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.521678 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" event={"ID":"b4de1cee-9308-4757-9033-021dddf0e53a","Type":"ContainerDied","Data":"5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397"} Jul 16 21:56:01.521915 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.521731 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq" event={"ID":"b4de1cee-9308-4757-9033-021dddf0e53a","Type":"ContainerDied","Data":"5952c4ef15b022d786bc3dac6e751a6552f557843bc9ac493424b3dff3f891f7"} Jul 16 21:56:01.521915 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.521757 2563 scope.go:117] "RemoveContainer" containerID="5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397" Jul 16 21:56:01.530314 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.530297 2563 scope.go:117] "RemoveContainer" containerID="5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397" Jul 16 21:56:01.530566 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:56:01.530548 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397\": container with ID starting with 5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397 not found: ID does not exist" containerID="5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397" Jul 16 21:56:01.530625 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.530574 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397"} err="failed to get container status \"5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397\": rpc error: code = NotFound desc = could not find container \"5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397\": container with ID starting with 5bbde118d30f04c332f0aba8b645ce8a453a4eeb18151d568961dbad5f66a397 not found: ID does not exist" Jul 16 21:56:01.592476 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.592441 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq"] Jul 16 21:56:01.594236 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:01.594218 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-2pdkq"] Jul 16 21:56:03.241216 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:03.241186 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4de1cee-9308-4757-9033-021dddf0e53a" path="/var/lib/kubelet/pods/b4de1cee-9308-4757-9033-021dddf0e53a/volumes" Jul 16 21:56:08.508743 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:08.508710 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6c6c96995f-mhmwt" Jul 16 21:56:43.308073 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.308030 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hrdbb"] Jul 16 21:56:43.308635 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.308582 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4de1cee-9308-4757-9033-021dddf0e53a" containerName="discovery" Jul 16 21:56:43.308635 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.308605 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4de1cee-9308-4757-9033-021dddf0e53a" containerName="discovery" Jul 16 21:56:43.308754 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.308707 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4de1cee-9308-4757-9033-021dddf0e53a" containerName="discovery" Jul 16 21:56:43.311667 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.311647 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" Jul 16 21:56:43.315100 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.315078 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-dphr6\"" Jul 16 21:56:43.315339 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.315325 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 21:56:43.315533 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.315519 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 21:56:43.334125 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.334100 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hrdbb"] Jul 16 21:56:43.375981 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.375956 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctk4d\" (UniqueName: \"kubernetes.io/projected/af77cfd6-8236-4059-a9ad-3d3d57c60efe-kube-api-access-ctk4d\") pod \"kuadrant-operator-catalog-hrdbb\" (UID: \"af77cfd6-8236-4059-a9ad-3d3d57c60efe\") " pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" Jul 16 21:56:43.477213 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.477184 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctk4d\" (UniqueName: \"kubernetes.io/projected/af77cfd6-8236-4059-a9ad-3d3d57c60efe-kube-api-access-ctk4d\") pod \"kuadrant-operator-catalog-hrdbb\" (UID: \"af77cfd6-8236-4059-a9ad-3d3d57c60efe\") " pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" Jul 16 21:56:43.486907 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.486879 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctk4d\" (UniqueName: \"kubernetes.io/projected/af77cfd6-8236-4059-a9ad-3d3d57c60efe-kube-api-access-ctk4d\") pod \"kuadrant-operator-catalog-hrdbb\" (UID: \"af77cfd6-8236-4059-a9ad-3d3d57c60efe\") " pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" Jul 16 21:56:43.583993 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.583927 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hrdbb"] Jul 16 21:56:43.584204 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.584190 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" Jul 16 21:56:43.721556 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.721474 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hrdbb"] Jul 16 21:56:43.723764 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:56:43.723728 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf77cfd6_8236_4059_a9ad_3d3d57c60efe.slice/crio-d0ea3fba4f5fbdf2f2d286f471d34539f3eaa38542b9d578781ea00e62c3ee34 WatchSource:0}: Error finding container d0ea3fba4f5fbdf2f2d286f471d34539f3eaa38542b9d578781ea00e62c3ee34: Status 404 returned error can't find the container with id d0ea3fba4f5fbdf2f2d286f471d34539f3eaa38542b9d578781ea00e62c3ee34 Jul 16 21:56:43.731044 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.731014 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" event={"ID":"af77cfd6-8236-4059-a9ad-3d3d57c60efe","Type":"ContainerStarted","Data":"d0ea3fba4f5fbdf2f2d286f471d34539f3eaa38542b9d578781ea00e62c3ee34"} Jul 16 21:56:43.793474 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.793448 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8jnqf"] Jul 16 21:56:43.797945 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.797930 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:43.804328 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.804300 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8jnqf"] Jul 16 21:56:43.880863 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.880839 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrfxc\" (UniqueName: \"kubernetes.io/projected/4b4b6cd2-b946-4036-9550-0500679dd755-kube-api-access-lrfxc\") pod \"kuadrant-operator-catalog-8jnqf\" (UID: \"4b4b6cd2-b946-4036-9550-0500679dd755\") " pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:43.981693 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.981664 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrfxc\" (UniqueName: \"kubernetes.io/projected/4b4b6cd2-b946-4036-9550-0500679dd755-kube-api-access-lrfxc\") pod \"kuadrant-operator-catalog-8jnqf\" (UID: \"4b4b6cd2-b946-4036-9550-0500679dd755\") " pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:43.990803 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:43.990777 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrfxc\" (UniqueName: \"kubernetes.io/projected/4b4b6cd2-b946-4036-9550-0500679dd755-kube-api-access-lrfxc\") pod \"kuadrant-operator-catalog-8jnqf\" (UID: \"4b4b6cd2-b946-4036-9550-0500679dd755\") " pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:44.108215 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:44.108183 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:44.272990 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:44.272958 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8jnqf"] Jul 16 21:56:44.276353 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:56:44.276326 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4b6cd2_b946_4036_9550_0500679dd755.slice/crio-8b9f4259e9f36448d402be37800ca6280921365e8e02dc2462c1c9b8c5185ff6 WatchSource:0}: Error finding container 8b9f4259e9f36448d402be37800ca6280921365e8e02dc2462c1c9b8c5185ff6: Status 404 returned error can't find the container with id 8b9f4259e9f36448d402be37800ca6280921365e8e02dc2462c1c9b8c5185ff6 Jul 16 21:56:44.738096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:44.738062 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" event={"ID":"4b4b6cd2-b946-4036-9550-0500679dd755","Type":"ContainerStarted","Data":"8b9f4259e9f36448d402be37800ca6280921365e8e02dc2462c1c9b8c5185ff6"} Jul 16 21:56:45.214430 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:45.214401 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 21:56:45.215053 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:45.215033 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 21:56:45.217731 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:45.217713 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 21:56:45.218215 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:45.218195 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 21:56:46.750800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:46.750761 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" event={"ID":"4b4b6cd2-b946-4036-9550-0500679dd755","Type":"ContainerStarted","Data":"b83d1b8eaa68c9d933f585525de9578b80d0e0640d8c444fde76f9c896f85a7d"} Jul 16 21:56:46.752230 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:46.752207 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" event={"ID":"af77cfd6-8236-4059-a9ad-3d3d57c60efe","Type":"ContainerStarted","Data":"61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e"} Jul 16 21:56:46.752316 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:46.752290 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" podUID="af77cfd6-8236-4059-a9ad-3d3d57c60efe" containerName="registry-server" containerID="cri-o://61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e" gracePeriod=2 Jul 16 21:56:46.782085 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:46.782041 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" podStartSLOduration=2.061326038 podStartE2EDuration="3.782027085s" totalImagesPullingTime="1.720701047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:56:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:56:46.780493054 +0000 UTC m=+602.113572104" watchObservedRunningTime="2026-07-16 21:56:46.782027085 +0000 UTC m=+602.115106115" Jul 16 21:56:46.992081 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:46.992059 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" Jul 16 21:56:47.112087 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.112062 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctk4d\" (UniqueName: \"kubernetes.io/projected/af77cfd6-8236-4059-a9ad-3d3d57c60efe-kube-api-access-ctk4d\") pod \"af77cfd6-8236-4059-a9ad-3d3d57c60efe\" (UID: \"af77cfd6-8236-4059-a9ad-3d3d57c60efe\") " Jul 16 21:56:47.114198 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.114172 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af77cfd6-8236-4059-a9ad-3d3d57c60efe-kube-api-access-ctk4d" (OuterVolumeSpecName: "kube-api-access-ctk4d") pod "af77cfd6-8236-4059-a9ad-3d3d57c60efe" (UID: "af77cfd6-8236-4059-a9ad-3d3d57c60efe"). InnerVolumeSpecName "kube-api-access-ctk4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:56:47.213477 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.213442 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctk4d\" (UniqueName: \"kubernetes.io/projected/af77cfd6-8236-4059-a9ad-3d3d57c60efe-kube-api-access-ctk4d\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:56:47.759209 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.759170 2563 generic.go:356] "Generic (PLEG): container finished" podID="af77cfd6-8236-4059-a9ad-3d3d57c60efe" containerID="61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e" exitCode=0 Jul 16 21:56:47.759679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.759226 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" Jul 16 21:56:47.759679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.759239 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" event={"ID":"af77cfd6-8236-4059-a9ad-3d3d57c60efe","Type":"ContainerDied","Data":"61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e"} Jul 16 21:56:47.759679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.759320 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hrdbb" event={"ID":"af77cfd6-8236-4059-a9ad-3d3d57c60efe","Type":"ContainerDied","Data":"d0ea3fba4f5fbdf2f2d286f471d34539f3eaa38542b9d578781ea00e62c3ee34"} Jul 16 21:56:47.759679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.759352 2563 scope.go:117] "RemoveContainer" containerID="61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e" Jul 16 21:56:47.768271 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.768253 2563 scope.go:117] "RemoveContainer" containerID="61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e" Jul 16 21:56:47.768554 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:56:47.768532 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e\": container with ID starting with 61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e not found: ID does not exist" containerID="61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e" Jul 16 21:56:47.768674 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.768558 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e"} err="failed to get container status \"61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e\": rpc error: code = NotFound desc = could not find container \"61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e\": container with ID starting with 61c461892c7f9c6d3479e860f3ba310848d32b227d33d622c9340b1f7315ce5e not found: ID does not exist" Jul 16 21:56:47.783316 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.783295 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hrdbb"] Jul 16 21:56:47.788725 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:47.788705 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hrdbb"] Jul 16 21:56:49.240981 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:49.240945 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af77cfd6-8236-4059-a9ad-3d3d57c60efe" path="/var/lib/kubelet/pods/af77cfd6-8236-4059-a9ad-3d3d57c60efe/volumes" Jul 16 21:56:54.108782 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:54.108745 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:54.109305 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:54.108806 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:54.132036 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:54.132013 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:54.817753 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:54.817726 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-8jnqf" Jul 16 21:56:58.637890 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.637852 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c"] Jul 16 21:56:58.638291 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.638259 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af77cfd6-8236-4059-a9ad-3d3d57c60efe" containerName="registry-server" Jul 16 21:56:58.638291 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.638271 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af77cfd6-8236-4059-a9ad-3d3d57c60efe" containerName="registry-server" Jul 16 21:56:58.638362 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.638334 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af77cfd6-8236-4059-a9ad-3d3d57c60efe" containerName="registry-server" Jul 16 21:56:58.642852 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.642833 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.646811 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.646792 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pm9j9\"" Jul 16 21:56:58.659828 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.659804 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c"] Jul 16 21:56:58.709647 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.709618 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.709647 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.709649 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.709908 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.709702 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gcgg\" (UniqueName: \"kubernetes.io/projected/3d05f907-88be-4d4b-83ab-fa075e136697-kube-api-access-4gcgg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.810956 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.810918 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.810956 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.810956 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.811252 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.811009 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gcgg\" (UniqueName: \"kubernetes.io/projected/3d05f907-88be-4d4b-83ab-fa075e136697-kube-api-access-4gcgg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.811433 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.811409 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.811510 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.811480 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.822360 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.822336 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gcgg\" (UniqueName: \"kubernetes.io/projected/3d05f907-88be-4d4b-83ab-fa075e136697-kube-api-access-4gcgg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:58.952359 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:58.952268 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:56:59.084450 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.084425 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c"] Jul 16 21:56:59.086392 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:56:59.086362 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d05f907_88be_4d4b_83ab_fa075e136697.slice/crio-1c078ae2e9acedc721c69eaac392fc6cf740eed2445d80dc3fa17b51b07ed826 WatchSource:0}: Error finding container 1c078ae2e9acedc721c69eaac392fc6cf740eed2445d80dc3fa17b51b07ed826: Status 404 returned error can't find the container with id 1c078ae2e9acedc721c69eaac392fc6cf740eed2445d80dc3fa17b51b07ed826 Jul 16 21:56:59.270601 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.270527 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74"] Jul 16 21:56:59.274105 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.274088 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.287717 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.287688 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74"] Jul 16 21:56:59.416420 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.416385 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.416420 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.416426 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.416642 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.416493 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n54rd\" (UniqueName: \"kubernetes.io/projected/46e10611-d199-40ff-8bdf-c2521f06cd97-kube-api-access-n54rd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.517227 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.517195 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.517227 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.517233 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.517475 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.517267 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n54rd\" (UniqueName: \"kubernetes.io/projected/46e10611-d199-40ff-8bdf-c2521f06cd97-kube-api-access-n54rd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.517642 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.517620 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.517642 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.517635 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.528191 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.528115 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n54rd\" (UniqueName: \"kubernetes.io/projected/46e10611-d199-40ff-8bdf-c2521f06cd97-kube-api-access-n54rd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.583121 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.583085 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:56:59.619039 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.619008 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9"] Jul 16 21:56:59.623801 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.623778 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.633631 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.633603 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9"] Jul 16 21:56:59.719733 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.719692 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.719733 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.719735 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/1b892c73-a13d-4248-9ce0-64e8024e4674-kube-api-access-plxcv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.720287 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.719858 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.723763 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.723741 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74"] Jul 16 21:56:59.725655 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:56:59.725624 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e10611_d199_40ff_8bdf_c2521f06cd97.slice/crio-7170311adf8994573175244b85a8116290b917d58f6ae088cb799cd9d2257321 WatchSource:0}: Error finding container 7170311adf8994573175244b85a8116290b917d58f6ae088cb799cd9d2257321: Status 404 returned error can't find the container with id 7170311adf8994573175244b85a8116290b917d58f6ae088cb799cd9d2257321 Jul 16 21:56:59.820859 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.820832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.821060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.820897 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.821060 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.820923 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/1b892c73-a13d-4248-9ce0-64e8024e4674-kube-api-access-plxcv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.821320 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.821296 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.821320 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.821310 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.823758 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.823732 2563 generic.go:356] "Generic (PLEG): container finished" podID="3d05f907-88be-4d4b-83ab-fa075e136697" containerID="e986ee8b057adda5d88002e7bd6afa0a9ebd0556c5afb37cce51a0cefa6e8f9d" exitCode=0 Jul 16 21:56:59.823906 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.823814 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" event={"ID":"3d05f907-88be-4d4b-83ab-fa075e136697","Type":"ContainerDied","Data":"e986ee8b057adda5d88002e7bd6afa0a9ebd0556c5afb37cce51a0cefa6e8f9d"} Jul 16 21:56:59.823906 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.823876 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" event={"ID":"3d05f907-88be-4d4b-83ab-fa075e136697","Type":"ContainerStarted","Data":"1c078ae2e9acedc721c69eaac392fc6cf740eed2445d80dc3fa17b51b07ed826"} Jul 16 21:56:59.824363 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.824346 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 21:56:59.825800 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.825777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" event={"ID":"46e10611-d199-40ff-8bdf-c2521f06cd97","Type":"ContainerStarted","Data":"496d40b6b64360485eadd1ef028ff05f8a36eb032b47753850639207e27426cb"} Jul 16 21:56:59.825878 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.825825 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" event={"ID":"46e10611-d199-40ff-8bdf-c2521f06cd97","Type":"ContainerStarted","Data":"7170311adf8994573175244b85a8116290b917d58f6ae088cb799cd9d2257321"} Jul 16 21:56:59.831678 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.831654 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/1b892c73-a13d-4248-9ce0-64e8024e4674-kube-api-access-plxcv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:56:59.934468 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:56:59.934434 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:57:00.042664 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.042578 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl"] Jul 16 21:57:00.047608 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.047583 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.074179 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.074116 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl"] Jul 16 21:57:00.094961 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.094936 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9"] Jul 16 21:57:00.097174 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:57:00.097132 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b892c73_a13d_4248_9ce0_64e8024e4674.slice/crio-8d2abfb3a34d046b5973f4febb411bc5f867b143cf6f4ee34f6cc70427ead206 WatchSource:0}: Error finding container 8d2abfb3a34d046b5973f4febb411bc5f867b143cf6f4ee34f6cc70427ead206: Status 404 returned error can't find the container with id 8d2abfb3a34d046b5973f4febb411bc5f867b143cf6f4ee34f6cc70427ead206 Jul 16 21:57:00.123685 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.123658 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.123793 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.123705 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndv22\" (UniqueName: \"kubernetes.io/projected/b6fc0945-783d-4dd7-b85e-57ed897d55c5-kube-api-access-ndv22\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.123855 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.123826 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.225318 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.225288 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.225438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.225328 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndv22\" (UniqueName: \"kubernetes.io/projected/b6fc0945-783d-4dd7-b85e-57ed897d55c5-kube-api-access-ndv22\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.225438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.225377 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.225695 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.225676 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.225730 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.225711 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.235707 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.235680 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndv22\" (UniqueName: \"kubernetes.io/projected/b6fc0945-783d-4dd7-b85e-57ed897d55c5-kube-api-access-ndv22\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.357500 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.357473 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:00.610964 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.610940 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl"] Jul 16 21:57:00.650988 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:57:00.650957 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6fc0945_783d_4dd7_b85e_57ed897d55c5.slice/crio-4dccf1a79d25ce07d86c0c4a727ac986fc77d6c044292d450e674ee27aaa58e4 WatchSource:0}: Error finding container 4dccf1a79d25ce07d86c0c4a727ac986fc77d6c044292d450e674ee27aaa58e4: Status 404 returned error can't find the container with id 4dccf1a79d25ce07d86c0c4a727ac986fc77d6c044292d450e674ee27aaa58e4 Jul 16 21:57:00.833510 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.833479 2563 generic.go:356] "Generic (PLEG): container finished" podID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerID="496d40b6b64360485eadd1ef028ff05f8a36eb032b47753850639207e27426cb" exitCode=0 Jul 16 21:57:00.833967 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.833565 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" event={"ID":"46e10611-d199-40ff-8bdf-c2521f06cd97","Type":"ContainerDied","Data":"496d40b6b64360485eadd1ef028ff05f8a36eb032b47753850639207e27426cb"} Jul 16 21:57:00.835342 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.835316 2563 generic.go:356] "Generic (PLEG): container finished" podID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerID="5ec03405607e9322ecf642c165946343d59a59c3c6237c2720e91cf95dcaf22e" exitCode=0 Jul 16 21:57:00.835437 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.835355 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" event={"ID":"b6fc0945-783d-4dd7-b85e-57ed897d55c5","Type":"ContainerDied","Data":"5ec03405607e9322ecf642c165946343d59a59c3c6237c2720e91cf95dcaf22e"} Jul 16 21:57:00.835437 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.835394 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" event={"ID":"b6fc0945-783d-4dd7-b85e-57ed897d55c5","Type":"ContainerStarted","Data":"4dccf1a79d25ce07d86c0c4a727ac986fc77d6c044292d450e674ee27aaa58e4"} Jul 16 21:57:00.837465 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.837444 2563 generic.go:356] "Generic (PLEG): container finished" podID="3d05f907-88be-4d4b-83ab-fa075e136697" containerID="602b74438a07f59ee3cd2d6dbf55d5e77992b4fc9440a1d049999bc77eaaa06c" exitCode=0 Jul 16 21:57:00.837561 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.837522 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" event={"ID":"3d05f907-88be-4d4b-83ab-fa075e136697","Type":"ContainerDied","Data":"602b74438a07f59ee3cd2d6dbf55d5e77992b4fc9440a1d049999bc77eaaa06c"} Jul 16 21:57:00.839136 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.839119 2563 generic.go:356] "Generic (PLEG): container finished" podID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerID="1398f38d7d1ab9eb32dc30b06040a772741807aba6b7af2e26ef3785df1618c1" exitCode=0 Jul 16 21:57:00.839227 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.839187 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" event={"ID":"1b892c73-a13d-4248-9ce0-64e8024e4674","Type":"ContainerDied","Data":"1398f38d7d1ab9eb32dc30b06040a772741807aba6b7af2e26ef3785df1618c1"} Jul 16 21:57:00.839227 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:00.839214 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" event={"ID":"1b892c73-a13d-4248-9ce0-64e8024e4674","Type":"ContainerStarted","Data":"8d2abfb3a34d046b5973f4febb411bc5f867b143cf6f4ee34f6cc70427ead206"} Jul 16 21:57:01.847281 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:01.847252 2563 generic.go:356] "Generic (PLEG): container finished" podID="3d05f907-88be-4d4b-83ab-fa075e136697" containerID="ba42ca61c52d64e33fc3c30df4ba202b2a3f3645e350f73938a8f1a27295bd16" exitCode=0 Jul 16 21:57:01.847708 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:01.847331 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" event={"ID":"3d05f907-88be-4d4b-83ab-fa075e136697","Type":"ContainerDied","Data":"ba42ca61c52d64e33fc3c30df4ba202b2a3f3645e350f73938a8f1a27295bd16"} Jul 16 21:57:01.849255 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:01.849226 2563 generic.go:356] "Generic (PLEG): container finished" podID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerID="157c468d5a23630c90a00dc1586b41de5317b7b7bb5a7b9937fcfba78324a295" exitCode=0 Jul 16 21:57:01.849359 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:01.849309 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" event={"ID":"1b892c73-a13d-4248-9ce0-64e8024e4674","Type":"ContainerDied","Data":"157c468d5a23630c90a00dc1586b41de5317b7b7bb5a7b9937fcfba78324a295"} Jul 16 21:57:01.851078 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:01.851058 2563 generic.go:356] "Generic (PLEG): container finished" podID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerID="3227c3b1710df52a2f7df422e955f650393b7c3f353ed9e0d27d098d7151a334" exitCode=0 Jul 16 21:57:01.851189 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:01.851083 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" event={"ID":"46e10611-d199-40ff-8bdf-c2521f06cd97","Type":"ContainerDied","Data":"3227c3b1710df52a2f7df422e955f650393b7c3f353ed9e0d27d098d7151a334"} Jul 16 21:57:02.862646 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:02.862612 2563 generic.go:356] "Generic (PLEG): container finished" podID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerID="4e8f8c00b3b1a80e6057042b71866080d1fce2c9ed0f5fba65c736473516b984" exitCode=0 Jul 16 21:57:02.863086 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:02.862699 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" event={"ID":"1b892c73-a13d-4248-9ce0-64e8024e4674","Type":"ContainerDied","Data":"4e8f8c00b3b1a80e6057042b71866080d1fce2c9ed0f5fba65c736473516b984"} Jul 16 21:57:02.864518 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:02.864498 2563 generic.go:356] "Generic (PLEG): container finished" podID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerID="49fd0909790e33ba301420b7c8cceb40bdeee73d0a4c2f8861fbc2a6bd4a7b93" exitCode=0 Jul 16 21:57:02.864632 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:02.864575 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" event={"ID":"46e10611-d199-40ff-8bdf-c2521f06cd97","Type":"ContainerDied","Data":"49fd0909790e33ba301420b7c8cceb40bdeee73d0a4c2f8861fbc2a6bd4a7b93"} Jul 16 21:57:02.866187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:02.866165 2563 generic.go:356] "Generic (PLEG): container finished" podID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerID="d410bf7497b8299077addfcfa56c86e81a8f1fc1021f0afaf8e59fb60510d8ba" exitCode=0 Jul 16 21:57:02.866279 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:02.866248 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" event={"ID":"b6fc0945-783d-4dd7-b85e-57ed897d55c5","Type":"ContainerDied","Data":"d410bf7497b8299077addfcfa56c86e81a8f1fc1021f0afaf8e59fb60510d8ba"} Jul 16 21:57:03.004297 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.004265 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:57:03.151196 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.151163 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-bundle\") pod \"3d05f907-88be-4d4b-83ab-fa075e136697\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " Jul 16 21:57:03.151354 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.151212 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gcgg\" (UniqueName: \"kubernetes.io/projected/3d05f907-88be-4d4b-83ab-fa075e136697-kube-api-access-4gcgg\") pod \"3d05f907-88be-4d4b-83ab-fa075e136697\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " Jul 16 21:57:03.151354 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.151246 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-util\") pod \"3d05f907-88be-4d4b-83ab-fa075e136697\" (UID: \"3d05f907-88be-4d4b-83ab-fa075e136697\") " Jul 16 21:57:03.151660 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.151635 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-bundle" (OuterVolumeSpecName: "bundle") pod "3d05f907-88be-4d4b-83ab-fa075e136697" (UID: "3d05f907-88be-4d4b-83ab-fa075e136697"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:03.153390 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.153358 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d05f907-88be-4d4b-83ab-fa075e136697-kube-api-access-4gcgg" (OuterVolumeSpecName: "kube-api-access-4gcgg") pod "3d05f907-88be-4d4b-83ab-fa075e136697" (UID: "3d05f907-88be-4d4b-83ab-fa075e136697"). InnerVolumeSpecName "kube-api-access-4gcgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:57:03.156462 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.156441 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-util" (OuterVolumeSpecName: "util") pod "3d05f907-88be-4d4b-83ab-fa075e136697" (UID: "3d05f907-88be-4d4b-83ab-fa075e136697"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:03.252913 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.252892 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:03.253007 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.252913 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gcgg\" (UniqueName: \"kubernetes.io/projected/3d05f907-88be-4d4b-83ab-fa075e136697-kube-api-access-4gcgg\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:03.253007 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.252924 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d05f907-88be-4d4b-83ab-fa075e136697-util\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:03.874598 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.874570 2563 generic.go:356] "Generic (PLEG): container finished" podID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerID="56515936ca811b1f6399e674efc064b640a049864adc2fb98fb08eef5d308bb6" exitCode=0 Jul 16 21:57:03.874986 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.874663 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" event={"ID":"b6fc0945-783d-4dd7-b85e-57ed897d55c5","Type":"ContainerDied","Data":"56515936ca811b1f6399e674efc064b640a049864adc2fb98fb08eef5d308bb6"} Jul 16 21:57:03.876539 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.876510 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" event={"ID":"3d05f907-88be-4d4b-83ab-fa075e136697","Type":"ContainerDied","Data":"1c078ae2e9acedc721c69eaac392fc6cf740eed2445d80dc3fa17b51b07ed826"} Jul 16 21:57:03.876648 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.876557 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c078ae2e9acedc721c69eaac392fc6cf740eed2445d80dc3fa17b51b07ed826" Jul 16 21:57:03.876917 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:03.876825 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c" Jul 16 21:57:04.012839 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.012815 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:57:04.068386 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.068358 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:57:04.160426 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.160400 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/1b892c73-a13d-4248-9ce0-64e8024e4674-kube-api-access-plxcv\") pod \"1b892c73-a13d-4248-9ce0-64e8024e4674\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " Jul 16 21:57:04.160604 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.160436 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-bundle\") pod \"1b892c73-a13d-4248-9ce0-64e8024e4674\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " Jul 16 21:57:04.160604 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.160454 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-util\") pod \"1b892c73-a13d-4248-9ce0-64e8024e4674\" (UID: \"1b892c73-a13d-4248-9ce0-64e8024e4674\") " Jul 16 21:57:04.160604 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.160497 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n54rd\" (UniqueName: \"kubernetes.io/projected/46e10611-d199-40ff-8bdf-c2521f06cd97-kube-api-access-n54rd\") pod \"46e10611-d199-40ff-8bdf-c2521f06cd97\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " Jul 16 21:57:04.160604 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.160529 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-bundle\") pod \"46e10611-d199-40ff-8bdf-c2521f06cd97\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " Jul 16 21:57:04.160604 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.160547 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-util\") pod \"46e10611-d199-40ff-8bdf-c2521f06cd97\" (UID: \"46e10611-d199-40ff-8bdf-c2521f06cd97\") " Jul 16 21:57:04.161155 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.161081 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-bundle" (OuterVolumeSpecName: "bundle") pod "46e10611-d199-40ff-8bdf-c2521f06cd97" (UID: "46e10611-d199-40ff-8bdf-c2521f06cd97"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:04.161277 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.161225 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-bundle" (OuterVolumeSpecName: "bundle") pod "1b892c73-a13d-4248-9ce0-64e8024e4674" (UID: "1b892c73-a13d-4248-9ce0-64e8024e4674"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:04.162635 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.162613 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e10611-d199-40ff-8bdf-c2521f06cd97-kube-api-access-n54rd" (OuterVolumeSpecName: "kube-api-access-n54rd") pod "46e10611-d199-40ff-8bdf-c2521f06cd97" (UID: "46e10611-d199-40ff-8bdf-c2521f06cd97"). InnerVolumeSpecName "kube-api-access-n54rd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:57:04.162836 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.162823 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b892c73-a13d-4248-9ce0-64e8024e4674-kube-api-access-plxcv" (OuterVolumeSpecName: "kube-api-access-plxcv") pod "1b892c73-a13d-4248-9ce0-64e8024e4674" (UID: "1b892c73-a13d-4248-9ce0-64e8024e4674"). InnerVolumeSpecName "kube-api-access-plxcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:57:04.165676 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.165634 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-util" (OuterVolumeSpecName: "util") pod "1b892c73-a13d-4248-9ce0-64e8024e4674" (UID: "1b892c73-a13d-4248-9ce0-64e8024e4674"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:04.166225 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.166205 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-util" (OuterVolumeSpecName: "util") pod "46e10611-d199-40ff-8bdf-c2521f06cd97" (UID: "46e10611-d199-40ff-8bdf-c2521f06cd97"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:04.261692 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.261644 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n54rd\" (UniqueName: \"kubernetes.io/projected/46e10611-d199-40ff-8bdf-c2521f06cd97-kube-api-access-n54rd\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:04.261692 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.261689 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:04.261692 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.261700 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/46e10611-d199-40ff-8bdf-c2521f06cd97-util\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:04.261966 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.261711 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/1b892c73-a13d-4248-9ce0-64e8024e4674-kube-api-access-plxcv\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:04.261966 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.261721 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:04.261966 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.261729 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b892c73-a13d-4248-9ce0-64e8024e4674-util\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:04.885363 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.885338 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" Jul 16 21:57:04.885363 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.885347 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9" event={"ID":"1b892c73-a13d-4248-9ce0-64e8024e4674","Type":"ContainerDied","Data":"8d2abfb3a34d046b5973f4febb411bc5f867b143cf6f4ee34f6cc70427ead206"} Jul 16 21:57:04.885823 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.885394 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d2abfb3a34d046b5973f4febb411bc5f867b143cf6f4ee34f6cc70427ead206" Jul 16 21:57:04.887343 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.887308 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" event={"ID":"46e10611-d199-40ff-8bdf-c2521f06cd97","Type":"ContainerDied","Data":"7170311adf8994573175244b85a8116290b917d58f6ae088cb799cd9d2257321"} Jul 16 21:57:04.887343 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.887344 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7170311adf8994573175244b85a8116290b917d58f6ae088cb799cd9d2257321" Jul 16 21:57:04.887533 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:04.887313 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74" Jul 16 21:57:05.017573 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.017551 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:05.170237 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.170208 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndv22\" (UniqueName: \"kubernetes.io/projected/b6fc0945-783d-4dd7-b85e-57ed897d55c5-kube-api-access-ndv22\") pod \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " Jul 16 21:57:05.170408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.170251 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-bundle\") pod \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " Jul 16 21:57:05.170408 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.170321 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-util\") pod \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\" (UID: \"b6fc0945-783d-4dd7-b85e-57ed897d55c5\") " Jul 16 21:57:05.170780 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.170758 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-bundle" (OuterVolumeSpecName: "bundle") pod "b6fc0945-783d-4dd7-b85e-57ed897d55c5" (UID: "b6fc0945-783d-4dd7-b85e-57ed897d55c5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:05.172356 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.172333 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6fc0945-783d-4dd7-b85e-57ed897d55c5-kube-api-access-ndv22" (OuterVolumeSpecName: "kube-api-access-ndv22") pod "b6fc0945-783d-4dd7-b85e-57ed897d55c5" (UID: "b6fc0945-783d-4dd7-b85e-57ed897d55c5"). InnerVolumeSpecName "kube-api-access-ndv22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:57:05.175681 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.175657 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-util" (OuterVolumeSpecName: "util") pod "b6fc0945-783d-4dd7-b85e-57ed897d55c5" (UID: "b6fc0945-783d-4dd7-b85e-57ed897d55c5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:05.271401 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.271374 2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-util\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:05.271538 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.271421 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndv22\" (UniqueName: \"kubernetes.io/projected/b6fc0945-783d-4dd7-b85e-57ed897d55c5-kube-api-access-ndv22\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:05.271538 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.271437 2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6fc0945-783d-4dd7-b85e-57ed897d55c5-bundle\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:05.894945 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.894922 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" Jul 16 21:57:05.895333 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.894911 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl" event={"ID":"b6fc0945-783d-4dd7-b85e-57ed897d55c5","Type":"ContainerDied","Data":"4dccf1a79d25ce07d86c0c4a727ac986fc77d6c044292d450e674ee27aaa58e4"} Jul 16 21:57:05.895333 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:05.895043 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4dccf1a79d25ce07d86c0c4a727ac986fc77d6c044292d450e674ee27aaa58e4" Jul 16 21:57:23.765566 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765533 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2"] Jul 16 21:57:23.765928 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765888 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d05f907-88be-4d4b-83ab-fa075e136697" containerName="extract" Jul 16 21:57:23.765928 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765900 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d05f907-88be-4d4b-83ab-fa075e136697" containerName="extract" Jul 16 21:57:23.765928 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765915 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerName="pull" Jul 16 21:57:23.765928 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765920 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerName="pull" Jul 16 21:57:23.765928 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765928 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerName="util" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765941 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerName="util" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765951 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerName="util" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765957 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerName="util" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765963 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerName="util" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765968 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerName="util" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765974 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerName="pull" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765979 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerName="pull" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765984 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerName="extract" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765990 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerName="extract" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.765997 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerName="extract" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766001 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerName="extract" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766014 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d05f907-88be-4d4b-83ab-fa075e136697" containerName="util" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766019 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d05f907-88be-4d4b-83ab-fa075e136697" containerName="util" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766026 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerName="pull" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766031 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerName="pull" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766037 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerName="extract" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766042 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerName="extract" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766049 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d05f907-88be-4d4b-83ab-fa075e136697" containerName="pull" Jul 16 21:57:23.766096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766054 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d05f907-88be-4d4b-83ab-fa075e136697" containerName="pull" Jul 16 21:57:23.766672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766107 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="46e10611-d199-40ff-8bdf-c2521f06cd97" containerName="extract" Jul 16 21:57:23.766672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766115 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6fc0945-783d-4dd7-b85e-57ed897d55c5" containerName="extract" Jul 16 21:57:23.766672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766123 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d05f907-88be-4d4b-83ab-fa075e136697" containerName="extract" Jul 16 21:57:23.766672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.766131 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b892c73-a13d-4248-9ce0-64e8024e4674" containerName="extract" Jul 16 21:57:23.773774 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.773758 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:23.776921 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.776901 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ld9cr\"" Jul 16 21:57:23.792723 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.792701 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2"] Jul 16 21:57:23.830395 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.830373 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9qt6\" (UniqueName: \"kubernetes.io/projected/341234a9-4fae-47f2-8af9-d0a53103c7ff-kube-api-access-j9qt6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jrrg2\" (UID: \"341234a9-4fae-47f2-8af9-d0a53103c7ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:23.830497 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.830479 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341234a9-4fae-47f2-8af9-d0a53103c7ff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jrrg2\" (UID: \"341234a9-4fae-47f2-8af9-d0a53103c7ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:23.931577 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.931546 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341234a9-4fae-47f2-8af9-d0a53103c7ff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jrrg2\" (UID: \"341234a9-4fae-47f2-8af9-d0a53103c7ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:23.931701 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.931584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9qt6\" (UniqueName: \"kubernetes.io/projected/341234a9-4fae-47f2-8af9-d0a53103c7ff-kube-api-access-j9qt6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jrrg2\" (UID: \"341234a9-4fae-47f2-8af9-d0a53103c7ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:23.931939 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.931917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341234a9-4fae-47f2-8af9-d0a53103c7ff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jrrg2\" (UID: \"341234a9-4fae-47f2-8af9-d0a53103c7ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:23.940934 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:23.940902 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9qt6\" (UniqueName: \"kubernetes.io/projected/341234a9-4fae-47f2-8af9-d0a53103c7ff-kube-api-access-j9qt6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jrrg2\" (UID: \"341234a9-4fae-47f2-8af9-d0a53103c7ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:24.084323 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:24.084241 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:24.234091 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:24.234058 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2"] Jul 16 21:57:24.238022 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:57:24.237992 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341234a9_4fae_47f2_8af9_d0a53103c7ff.slice/crio-5ff9fab269e1e647b376b8ebabceb9c19993dde3da13627756b7d9f60641151e WatchSource:0}: Error finding container 5ff9fab269e1e647b376b8ebabceb9c19993dde3da13627756b7d9f60641151e: Status 404 returned error can't find the container with id 5ff9fab269e1e647b376b8ebabceb9c19993dde3da13627756b7d9f60641151e Jul 16 21:57:25.007030 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:25.006993 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" event={"ID":"341234a9-4fae-47f2-8af9-d0a53103c7ff","Type":"ContainerStarted","Data":"5ff9fab269e1e647b376b8ebabceb9c19993dde3da13627756b7d9f60641151e"} Jul 16 21:57:27.320584 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:27.320551 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt"] Jul 16 21:57:27.324491 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:27.324466 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" Jul 16 21:57:27.327590 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:27.327564 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xflcq\"" Jul 16 21:57:27.333095 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:27.333062 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt"] Jul 16 21:57:27.360579 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:27.360546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc9t8\" (UniqueName: \"kubernetes.io/projected/284b00cf-79a6-4106-9b58-4672485328ae-kube-api-access-wc9t8\") pod \"authorino-operator-75cc8d5cb7-f2qmt\" (UID: \"284b00cf-79a6-4106-9b58-4672485328ae\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" Jul 16 21:57:27.461976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:27.461930 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc9t8\" (UniqueName: \"kubernetes.io/projected/284b00cf-79a6-4106-9b58-4672485328ae-kube-api-access-wc9t8\") pod \"authorino-operator-75cc8d5cb7-f2qmt\" (UID: \"284b00cf-79a6-4106-9b58-4672485328ae\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" Jul 16 21:57:27.474832 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:27.474805 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc9t8\" (UniqueName: \"kubernetes.io/projected/284b00cf-79a6-4106-9b58-4672485328ae-kube-api-access-wc9t8\") pod \"authorino-operator-75cc8d5cb7-f2qmt\" (UID: \"284b00cf-79a6-4106-9b58-4672485328ae\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" Jul 16 21:57:27.638642 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:27.638611 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" Jul 16 21:57:28.294020 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:28.293986 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt"] Jul 16 21:57:28.297636 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:57:28.297612 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod284b00cf_79a6_4106_9b58_4672485328ae.slice/crio-ebec10aed0806de9c20ab9e3ee312b86e39fc07e9674897838c755db769a1635 WatchSource:0}: Error finding container ebec10aed0806de9c20ab9e3ee312b86e39fc07e9674897838c755db769a1635: Status 404 returned error can't find the container with id ebec10aed0806de9c20ab9e3ee312b86e39fc07e9674897838c755db769a1635 Jul 16 21:57:29.043535 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:29.043497 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" event={"ID":"341234a9-4fae-47f2-8af9-d0a53103c7ff","Type":"ContainerStarted","Data":"ae365b9e468a28ce3dc5afdf57670d4c5c35d94383455670266e949de0977e35"} Jul 16 21:57:29.044298 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:29.044274 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:29.045418 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:29.045389 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" event={"ID":"284b00cf-79a6-4106-9b58-4672485328ae","Type":"ContainerStarted","Data":"ebec10aed0806de9c20ab9e3ee312b86e39fc07e9674897838c755db769a1635"} Jul 16 21:57:29.071811 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:29.071657 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" podStartSLOduration=2.108011971 podStartE2EDuration="6.071638404s" totalImagesPullingTime="3.963626433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:57:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:57:29.066889332 +0000 UTC m=+644.399968363" watchObservedRunningTime="2026-07-16 21:57:29.071638404 +0000 UTC m=+644.404717435" Jul 16 21:57:31.059098 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:31.059064 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" event={"ID":"284b00cf-79a6-4106-9b58-4672485328ae","Type":"ContainerStarted","Data":"100ace05faf34200c4d1ebe54fc7515e76218eefe38f4f43629804ced715d76b"} Jul 16 21:57:31.059494 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:31.059167 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" Jul 16 21:57:31.083308 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:31.083251 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" podStartSLOduration=1.812259503 podStartE2EDuration="4.083237988s" totalImagesPullingTime="2.270978485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:57:31.082921928 +0000 UTC m=+646.416000958" watchObservedRunningTime="2026-07-16 21:57:31.083237988 +0000 UTC m=+646.416317018" Jul 16 21:57:41.061847 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:41.061811 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:42.067501 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:42.067467 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-f2qmt" Jul 16 21:57:42.959923 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:42.959886 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2"] Jul 16 21:57:42.960103 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:42.960066 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" podUID="341234a9-4fae-47f2-8af9-d0a53103c7ff" containerName="manager" containerID="cri-o://ae365b9e468a28ce3dc5afdf57670d4c5c35d94383455670266e949de0977e35" gracePeriod=2 Jul 16 21:57:42.975444 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:42.975415 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2"] Jul 16 21:57:43.002255 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.002223 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r"] Jul 16 21:57:43.002599 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.002587 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341234a9-4fae-47f2-8af9-d0a53103c7ff" containerName="manager" Jul 16 21:57:43.002653 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.002601 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="341234a9-4fae-47f2-8af9-d0a53103c7ff" containerName="manager" Jul 16 21:57:43.002690 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.002672 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="341234a9-4fae-47f2-8af9-d0a53103c7ff" containerName="manager" Jul 16 21:57:43.005748 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.005729 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:43.010638 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:57:43.010605 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jrrg2\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" podUID="341234a9-4fae-47f2-8af9-d0a53103c7ff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:43.103150 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.103121 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mjb2\" (UniqueName: \"kubernetes.io/projected/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-kube-api-access-8mjb2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wzl9r\" (UID: \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:43.103489 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.103262 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wzl9r\" (UID: \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:43.125083 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.125046 2563 generic.go:356] "Generic (PLEG): container finished" podID="341234a9-4fae-47f2-8af9-d0a53103c7ff" containerID="ae365b9e468a28ce3dc5afdf57670d4c5c35d94383455670266e949de0977e35" exitCode=0 Jul 16 21:57:43.142247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.142217 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r"] Jul 16 21:57:43.204035 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.204005 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wzl9r\" (UID: \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:43.204176 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.204063 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mjb2\" (UniqueName: \"kubernetes.io/projected/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-kube-api-access-8mjb2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wzl9r\" (UID: \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:43.204373 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.204353 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wzl9r\" (UID: \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:43.217631 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.217563 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mjb2\" (UniqueName: \"kubernetes.io/projected/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-kube-api-access-8mjb2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wzl9r\" (UID: \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:43.221305 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.221284 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:43.224256 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:57:43.224230 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jrrg2\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" podUID="341234a9-4fae-47f2-8af9-d0a53103c7ff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:43.305313 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.305287 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341234a9-4fae-47f2-8af9-d0a53103c7ff-extensions-socket-volume\") pod \"341234a9-4fae-47f2-8af9-d0a53103c7ff\" (UID: \"341234a9-4fae-47f2-8af9-d0a53103c7ff\") " Jul 16 21:57:43.305423 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.305352 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9qt6\" (UniqueName: \"kubernetes.io/projected/341234a9-4fae-47f2-8af9-d0a53103c7ff-kube-api-access-j9qt6\") pod \"341234a9-4fae-47f2-8af9-d0a53103c7ff\" (UID: \"341234a9-4fae-47f2-8af9-d0a53103c7ff\") " Jul 16 21:57:43.305724 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.305701 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341234a9-4fae-47f2-8af9-d0a53103c7ff-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "341234a9-4fae-47f2-8af9-d0a53103c7ff" (UID: "341234a9-4fae-47f2-8af9-d0a53103c7ff"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:57:43.307315 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.307292 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/341234a9-4fae-47f2-8af9-d0a53103c7ff-kube-api-access-j9qt6" (OuterVolumeSpecName: "kube-api-access-j9qt6") pod "341234a9-4fae-47f2-8af9-d0a53103c7ff" (UID: "341234a9-4fae-47f2-8af9-d0a53103c7ff"). InnerVolumeSpecName "kube-api-access-j9qt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:57:43.350504 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.350482 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:43.407000 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.406966 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/341234a9-4fae-47f2-8af9-d0a53103c7ff-extensions-socket-volume\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:43.407000 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.407000 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9qt6\" (UniqueName: \"kubernetes.io/projected/341234a9-4fae-47f2-8af9-d0a53103c7ff-kube-api-access-j9qt6\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:57:43.495644 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:43.495617 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r"] Jul 16 21:57:43.496633 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:57:43.496607 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0276e228_ba92_4e48_8c1f_d3f15dd2a0a6.slice/crio-d1aa5df5cbca7e402d7abf26abfb9cc0cadefbb9f6b6eec3c787a9b3071a2ccd WatchSource:0}: Error finding container d1aa5df5cbca7e402d7abf26abfb9cc0cadefbb9f6b6eec3c787a9b3071a2ccd: Status 404 returned error can't find the container with id d1aa5df5cbca7e402d7abf26abfb9cc0cadefbb9f6b6eec3c787a9b3071a2ccd Jul 16 21:57:44.133582 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:44.133547 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" event={"ID":"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6","Type":"ContainerStarted","Data":"cd253674a8345138f531fc2a1adf71c888f801f8fe946740ff4f0f8abd73f31d"} Jul 16 21:57:44.134036 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:44.133605 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" event={"ID":"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6","Type":"ContainerStarted","Data":"d1aa5df5cbca7e402d7abf26abfb9cc0cadefbb9f6b6eec3c787a9b3071a2ccd"} Jul 16 21:57:44.134036 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:44.133753 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:57:44.134937 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:44.134919 2563 scope.go:117] "RemoveContainer" containerID="ae365b9e468a28ce3dc5afdf57670d4c5c35d94383455670266e949de0977e35" Jul 16 21:57:44.135061 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:44.134924 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jrrg2" Jul 16 21:57:44.163097 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:44.163055 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" podStartSLOduration=2.163043802 podStartE2EDuration="2.163043802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:57:44.159502662 +0000 UTC m=+659.492581692" watchObservedRunningTime="2026-07-16 21:57:44.163043802 +0000 UTC m=+659.496122833" Jul 16 21:57:45.241385 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:45.241357 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="341234a9-4fae-47f2-8af9-d0a53103c7ff" path="/var/lib/kubelet/pods/341234a9-4fae-47f2-8af9-d0a53103c7ff/volumes" Jul 16 21:57:55.142982 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:57:55.142948 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:58:10.128538 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.128503 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r"] Jul 16 21:58:10.129191 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.128791 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" podUID="0276e228-ba92-4e48-8c1f-d3f15dd2a0a6" containerName="manager" containerID="cri-o://cd253674a8345138f531fc2a1adf71c888f801f8fe946740ff4f0f8abd73f31d" gracePeriod=10 Jul 16 21:58:10.279585 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.279541 2563 generic.go:356] "Generic (PLEG): container finished" podID="0276e228-ba92-4e48-8c1f-d3f15dd2a0a6" containerID="cd253674a8345138f531fc2a1adf71c888f801f8fe946740ff4f0f8abd73f31d" exitCode=0 Jul 16 21:58:10.279736 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.279603 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" event={"ID":"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6","Type":"ContainerDied","Data":"cd253674a8345138f531fc2a1adf71c888f801f8fe946740ff4f0f8abd73f31d"} Jul 16 21:58:10.382367 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.382296 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:58:10.446392 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.446361 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mjb2\" (UniqueName: \"kubernetes.io/projected/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-kube-api-access-8mjb2\") pod \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\" (UID: \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\") " Jul 16 21:58:10.446544 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.446443 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-extensions-socket-volume\") pod \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\" (UID: \"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6\") " Jul 16 21:58:10.446797 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.446763 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0276e228-ba92-4e48-8c1f-d3f15dd2a0a6" (UID: "0276e228-ba92-4e48-8c1f-d3f15dd2a0a6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 21:58:10.448438 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.448410 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-kube-api-access-8mjb2" (OuterVolumeSpecName: "kube-api-access-8mjb2") pod "0276e228-ba92-4e48-8c1f-d3f15dd2a0a6" (UID: "0276e228-ba92-4e48-8c1f-d3f15dd2a0a6"). InnerVolumeSpecName "kube-api-access-8mjb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:58:10.547867 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.547836 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mjb2\" (UniqueName: \"kubernetes.io/projected/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-kube-api-access-8mjb2\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:58:10.547867 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:10.547861 2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6-extensions-socket-volume\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:58:11.286753 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:11.286722 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" event={"ID":"0276e228-ba92-4e48-8c1f-d3f15dd2a0a6","Type":"ContainerDied","Data":"d1aa5df5cbca7e402d7abf26abfb9cc0cadefbb9f6b6eec3c787a9b3071a2ccd"} Jul 16 21:58:11.286753 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:11.286742 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r" Jul 16 21:58:11.287261 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:11.286782 2563 scope.go:117] "RemoveContainer" containerID="cd253674a8345138f531fc2a1adf71c888f801f8fe946740ff4f0f8abd73f31d" Jul 16 21:58:11.308065 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:11.308037 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r"] Jul 16 21:58:11.314502 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:11.314477 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wzl9r"] Jul 16 21:58:13.240928 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:13.240894 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0276e228-ba92-4e48-8c1f-d3f15dd2a0a6" path="/var/lib/kubelet/pods/0276e228-ba92-4e48-8c1f-d3f15dd2a0a6/volumes" Jul 16 21:58:26.531923 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.531884 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc"] Jul 16 21:58:26.532404 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.532300 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0276e228-ba92-4e48-8c1f-d3f15dd2a0a6" containerName="manager" Jul 16 21:58:26.532404 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.532313 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0276e228-ba92-4e48-8c1f-d3f15dd2a0a6" containerName="manager" Jul 16 21:58:26.532404 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.532397 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0276e228-ba92-4e48-8c1f-d3f15dd2a0a6" containerName="manager" Jul 16 21:58:26.536741 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.536722 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.539575 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.539554 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 21:58:26.539688 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.539602 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-22mwd\"" Jul 16 21:58:26.546839 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.546816 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc"] Jul 16 21:58:26.587180 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587121 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.587318 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587182 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.587318 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587226 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.587318 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587260 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj8d6\" (UniqueName: \"kubernetes.io/projected/b3ba87a0-de8b-4043-999d-3204f369eaca-kube-api-access-rj8d6\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.587318 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587282 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.587318 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587310 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.587513 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587349 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3ba87a0-de8b-4043-999d-3204f369eaca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.587513 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.587513 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.587425 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688187 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688138 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688321 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688198 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688321 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688217 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688321 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688239 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj8d6\" (UniqueName: \"kubernetes.io/projected/b3ba87a0-de8b-4043-999d-3204f369eaca-kube-api-access-rj8d6\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688321 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688257 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688321 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688284 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688321 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688312 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3ba87a0-de8b-4043-999d-3204f369eaca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688627 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688344 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688627 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688365 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688740 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688642 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688810 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688755 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688810 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688767 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.688936 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.688862 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.689106 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.689084 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3ba87a0-de8b-4043-999d-3204f369eaca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.690674 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.690643 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.690930 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.690914 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.697473 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.697447 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3ba87a0-de8b-4043-999d-3204f369eaca-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.697824 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.697805 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj8d6\" (UniqueName: \"kubernetes.io/projected/b3ba87a0-de8b-4043-999d-3204f369eaca-kube-api-access-rj8d6\") pod \"maas-default-gateway-openshift-default-56654dcf7-tvknc\" (UID: \"b3ba87a0-de8b-4043-999d-3204f369eaca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.848109 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.848034 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:26.985475 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:26.985447 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc"] Jul 16 21:58:26.986240 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:58:26.986214 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ba87a0_de8b_4043_999d_3204f369eaca.slice/crio-278415d2d1f6f1ec4140d8f41eaa262c5daa26fdef85684e64601963b0aa38bc WatchSource:0}: Error finding container 278415d2d1f6f1ec4140d8f41eaa262c5daa26fdef85684e64601963b0aa38bc: Status 404 returned error can't find the container with id 278415d2d1f6f1ec4140d8f41eaa262c5daa26fdef85684e64601963b0aa38bc Jul 16 21:58:27.165447 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.165414 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66d4d88769-hlbxd"] Jul 16 21:58:27.170134 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.170113 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.174673 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.174652 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 21:58:27.175172 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.175137 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 21:58:27.175277 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.175178 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 21:58:27.176725 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.176689 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dbx5n\"" Jul 16 21:58:27.176725 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.176700 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 21:58:27.176868 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.176731 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 21:58:27.180605 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.180583 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 21:58:27.195825 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.195803 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d4d88769-hlbxd"] Jul 16 21:58:27.294419 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.294391 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-oauth-config\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.294587 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.294431 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-serving-cert\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.294587 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.294466 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-trusted-ca-bundle\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.294587 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.294506 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-config\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.294587 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.294522 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-service-ca\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.294587 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.294544 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-oauth-serving-cert\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.294785 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.294658 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdszx\" (UniqueName: \"kubernetes.io/projected/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-kube-api-access-zdszx\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.377272 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.377241 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" event={"ID":"b3ba87a0-de8b-4043-999d-3204f369eaca","Type":"ContainerStarted","Data":"278415d2d1f6f1ec4140d8f41eaa262c5daa26fdef85684e64601963b0aa38bc"} Jul 16 21:58:27.395672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.395646 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-oauth-config\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.395774 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.395685 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-serving-cert\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.395774 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.395707 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-trusted-ca-bundle\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.395774 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.395727 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-config\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.395774 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.395743 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-service-ca\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.395774 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.395757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-oauth-serving-cert\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.396015 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.395810 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdszx\" (UniqueName: \"kubernetes.io/projected/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-kube-api-access-zdszx\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.396554 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.396530 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-service-ca\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.396692 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.396648 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-oauth-serving-cert\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.396762 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.396691 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-config\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.396762 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.396718 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-trusted-ca-bundle\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.398275 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.398253 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-oauth-config\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.398373 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.398357 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-console-serving-cert\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.404615 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.404588 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdszx\" (UniqueName: \"kubernetes.io/projected/ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272-kube-api-access-zdszx\") pod \"console-66d4d88769-hlbxd\" (UID: \"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272\") " pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.481262 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.481205 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:27.624886 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:27.624849 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66d4d88769-hlbxd"] Jul 16 21:58:27.627818 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:58:27.627789 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3b4e03_9dd6_49ca_aebb_8df4cb9f4272.slice/crio-6a3ab40a0ab935113088f765a31a88b3dbbc015346a178e6817c5027f99add7b WatchSource:0}: Error finding container 6a3ab40a0ab935113088f765a31a88b3dbbc015346a178e6817c5027f99add7b: Status 404 returned error can't find the container with id 6a3ab40a0ab935113088f765a31a88b3dbbc015346a178e6817c5027f99add7b Jul 16 21:58:28.386374 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:28.386329 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d4d88769-hlbxd" event={"ID":"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272","Type":"ContainerStarted","Data":"270e371e057967e5e6d9b9ad3c6c17d7159dd5f939277327a511102447d90de4"} Jul 16 21:58:28.386574 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:28.386394 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66d4d88769-hlbxd" event={"ID":"ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272","Type":"ContainerStarted","Data":"6a3ab40a0ab935113088f765a31a88b3dbbc015346a178e6817c5027f99add7b"} Jul 16 21:58:28.405688 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:28.405639 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66d4d88769-hlbxd" podStartSLOduration=1.4056191199999999 podStartE2EDuration="1.40561912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:58:28.404902547 +0000 UTC m=+703.737981599" watchObservedRunningTime="2026-07-16 21:58:28.40561912 +0000 UTC m=+703.738698157" Jul 16 21:58:29.503675 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:29.503631 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 21:58:29.503997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:29.503705 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 21:58:29.503997 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:29.503736 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 21:58:30.401229 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:30.401190 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" event={"ID":"b3ba87a0-de8b-4043-999d-3204f369eaca","Type":"ContainerStarted","Data":"a7cafce5ede05179bced86dd3039ed0df8a4a759bd4aa18ba0e49b8323889aae"} Jul 16 21:58:30.430984 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:30.430938 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" podStartSLOduration=1.9157747349999998 podStartE2EDuration="4.430923555s" totalImagesPullingTime="2.51514882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:58:30.428276115 +0000 UTC m=+705.761355164" watchObservedRunningTime="2026-07-16 21:58:30.430923555 +0000 UTC m=+705.764002586" Jul 16 21:58:30.848727 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:30.848651 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:30.854254 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:30.854227 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:31.407542 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.407513 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:31.408484 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.408459 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 21:58:31.469791 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.469759 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hmwzt"] Jul 16 21:58:31.476238 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.476221 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:31.479442 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.479418 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 21:58:31.479580 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.479418 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pm9j9\"" Jul 16 21:58:31.481631 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.481610 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hmwzt"] Jul 16 21:58:31.534433 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.534406 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b03f7607-4067-4ba5-a951-174633a0f804-config-file\") pod \"limitador-limitador-64c94fd64f-hmwzt\" (UID: \"b03f7607-4067-4ba5-a951-174633a0f804\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:31.534589 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.534482 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7p6n\" (UniqueName: \"kubernetes.io/projected/b03f7607-4067-4ba5-a951-174633a0f804-kube-api-access-x7p6n\") pod \"limitador-limitador-64c94fd64f-hmwzt\" (UID: \"b03f7607-4067-4ba5-a951-174633a0f804\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:31.563935 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.563901 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hmwzt"] Jul 16 21:58:31.636094 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.636061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b03f7607-4067-4ba5-a951-174633a0f804-config-file\") pod \"limitador-limitador-64c94fd64f-hmwzt\" (UID: \"b03f7607-4067-4ba5-a951-174633a0f804\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:31.636269 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.636111 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7p6n\" (UniqueName: \"kubernetes.io/projected/b03f7607-4067-4ba5-a951-174633a0f804-kube-api-access-x7p6n\") pod \"limitador-limitador-64c94fd64f-hmwzt\" (UID: \"b03f7607-4067-4ba5-a951-174633a0f804\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:31.636745 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.636725 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b03f7607-4067-4ba5-a951-174633a0f804-config-file\") pod \"limitador-limitador-64c94fd64f-hmwzt\" (UID: \"b03f7607-4067-4ba5-a951-174633a0f804\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:31.646240 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.646207 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7p6n\" (UniqueName: \"kubernetes.io/projected/b03f7607-4067-4ba5-a951-174633a0f804-kube-api-access-x7p6n\") pod \"limitador-limitador-64c94fd64f-hmwzt\" (UID: \"b03f7607-4067-4ba5-a951-174633a0f804\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:31.789262 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.789135 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:31.928571 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:31.928540 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hmwzt"] Jul 16 21:58:31.929659 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:58:31.929632 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03f7607_4067_4ba5_a951_174633a0f804.slice/crio-4a63b2e3a2a357ea70c126da71e6c2be3c5dece8aea52f4656c203dd5f9c832d WatchSource:0}: Error finding container 4a63b2e3a2a357ea70c126da71e6c2be3c5dece8aea52f4656c203dd5f9c832d: Status 404 returned error can't find the container with id 4a63b2e3a2a357ea70c126da71e6c2be3c5dece8aea52f4656c203dd5f9c832d Jul 16 21:58:32.414375 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.414339 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" event={"ID":"b03f7607-4067-4ba5-a951-174633a0f804","Type":"ContainerStarted","Data":"4a63b2e3a2a357ea70c126da71e6c2be3c5dece8aea52f4656c203dd5f9c832d"} Jul 16 21:58:32.422247 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.422202 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hmlzk"] Jul 16 21:58:32.429021 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.428992 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hmlzk" Jul 16 21:58:32.429983 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.429960 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hmlzk"] Jul 16 21:58:32.431999 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.431980 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ftjlj\"" Jul 16 21:58:32.544558 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.544526 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-796vw\" (UniqueName: \"kubernetes.io/projected/64153789-804f-426a-83fa-765231521d22-kube-api-access-796vw\") pod \"authorino-75d556bd77-hmlzk\" (UID: \"64153789-804f-426a-83fa-765231521d22\") " pod="kuadrant-system/authorino-75d556bd77-hmlzk" Jul 16 21:58:32.646626 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.646120 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-796vw\" (UniqueName: \"kubernetes.io/projected/64153789-804f-426a-83fa-765231521d22-kube-api-access-796vw\") pod \"authorino-75d556bd77-hmlzk\" (UID: \"64153789-804f-426a-83fa-765231521d22\") " pod="kuadrant-system/authorino-75d556bd77-hmlzk" Jul 16 21:58:32.655817 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.655770 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-796vw\" (UniqueName: \"kubernetes.io/projected/64153789-804f-426a-83fa-765231521d22-kube-api-access-796vw\") pod \"authorino-75d556bd77-hmlzk\" (UID: \"64153789-804f-426a-83fa-765231521d22\") " pod="kuadrant-system/authorino-75d556bd77-hmlzk" Jul 16 21:58:32.740606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.740523 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hmlzk" Jul 16 21:58:32.926236 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:32.926205 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hmlzk"] Jul 16 21:58:32.927962 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:58:32.927932 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64153789_804f_426a_83fa_765231521d22.slice/crio-a19e1c196657c8dc609fc91846046db333e14764530527cee12ae233698a29a3 WatchSource:0}: Error finding container a19e1c196657c8dc609fc91846046db333e14764530527cee12ae233698a29a3: Status 404 returned error can't find the container with id a19e1c196657c8dc609fc91846046db333e14764530527cee12ae233698a29a3 Jul 16 21:58:33.423643 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:33.423602 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hmlzk" event={"ID":"64153789-804f-426a-83fa-765231521d22","Type":"ContainerStarted","Data":"a19e1c196657c8dc609fc91846046db333e14764530527cee12ae233698a29a3"} Jul 16 21:58:37.450577 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:37.450486 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" event={"ID":"b03f7607-4067-4ba5-a951-174633a0f804","Type":"ContainerStarted","Data":"6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523"} Jul 16 21:58:37.451223 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:37.450647 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:37.451979 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:37.451956 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hmlzk" event={"ID":"64153789-804f-426a-83fa-765231521d22","Type":"ContainerStarted","Data":"fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699"} Jul 16 21:58:37.473826 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:37.473776 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" podStartSLOduration=1.8318724849999999 podStartE2EDuration="6.473761766s" totalImagesPullingTime="4.641889281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:58:37.47144803 +0000 UTC m=+712.804527060" watchObservedRunningTime="2026-07-16 21:58:37.473761766 +0000 UTC m=+712.806840795" Jul 16 21:58:37.482085 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:37.482060 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:37.482213 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:37.482101 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:37.486729 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:37.486705 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:37.487864 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:37.487827 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hmlzk" podStartSLOduration=1.8346473319999999 podStartE2EDuration="5.487813562s" totalImagesPullingTime="3.65316623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:58:37.486835662 +0000 UTC m=+712.819914695" watchObservedRunningTime="2026-07-16 21:58:37.487813562 +0000 UTC m=+712.820892592" Jul 16 21:58:38.462026 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:38.461995 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66d4d88769-hlbxd" Jul 16 21:58:46.722341 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:46.722304 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hmwzt"] Jul 16 21:58:46.722787 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:46.722515 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" podUID="b03f7607-4067-4ba5-a951-174633a0f804" containerName="limitador" containerID="cri-o://6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523" gracePeriod=30 Jul 16 21:58:46.723313 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:46.723180 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:47.286672 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.286611 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:47.384044 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.384011 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b03f7607-4067-4ba5-a951-174633a0f804-config-file\") pod \"b03f7607-4067-4ba5-a951-174633a0f804\" (UID: \"b03f7607-4067-4ba5-a951-174633a0f804\") " Jul 16 21:58:47.384245 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.384052 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7p6n\" (UniqueName: \"kubernetes.io/projected/b03f7607-4067-4ba5-a951-174633a0f804-kube-api-access-x7p6n\") pod \"b03f7607-4067-4ba5-a951-174633a0f804\" (UID: \"b03f7607-4067-4ba5-a951-174633a0f804\") " Jul 16 21:58:47.384431 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.384404 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b03f7607-4067-4ba5-a951-174633a0f804-config-file" (OuterVolumeSpecName: "config-file") pod "b03f7607-4067-4ba5-a951-174633a0f804" (UID: "b03f7607-4067-4ba5-a951-174633a0f804"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 21:58:47.386117 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.386097 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b03f7607-4067-4ba5-a951-174633a0f804-kube-api-access-x7p6n" (OuterVolumeSpecName: "kube-api-access-x7p6n") pod "b03f7607-4067-4ba5-a951-174633a0f804" (UID: "b03f7607-4067-4ba5-a951-174633a0f804"). InnerVolumeSpecName "kube-api-access-x7p6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:58:47.485073 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.485044 2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b03f7607-4067-4ba5-a951-174633a0f804-config-file\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:58:47.485073 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.485073 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7p6n\" (UniqueName: \"kubernetes.io/projected/b03f7607-4067-4ba5-a951-174633a0f804-kube-api-access-x7p6n\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:58:47.508734 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.508700 2563 generic.go:356] "Generic (PLEG): container finished" podID="b03f7607-4067-4ba5-a951-174633a0f804" containerID="6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523" exitCode=0 Jul 16 21:58:47.508871 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.508758 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" Jul 16 21:58:47.508871 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.508785 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" event={"ID":"b03f7607-4067-4ba5-a951-174633a0f804","Type":"ContainerDied","Data":"6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523"} Jul 16 21:58:47.508871 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.508851 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-hmwzt" event={"ID":"b03f7607-4067-4ba5-a951-174633a0f804","Type":"ContainerDied","Data":"4a63b2e3a2a357ea70c126da71e6c2be3c5dece8aea52f4656c203dd5f9c832d"} Jul 16 21:58:47.508969 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.508887 2563 scope.go:117] "RemoveContainer" containerID="6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523" Jul 16 21:58:47.518134 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.518111 2563 scope.go:117] "RemoveContainer" containerID="6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523" Jul 16 21:58:47.518415 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:58:47.518397 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523\": container with ID starting with 6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523 not found: ID does not exist" containerID="6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523" Jul 16 21:58:47.518488 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.518428 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523"} err="failed to get container status \"6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523\": rpc error: code = NotFound desc = could not find container \"6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523\": container with ID starting with 6937304e5b028d0665a076ed646a1771bd2a1b75248b93446a27bdc914ee5523 not found: ID does not exist" Jul 16 21:58:47.532680 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.532641 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hmwzt"] Jul 16 21:58:47.535021 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:47.535004 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-hmwzt"] Jul 16 21:58:49.241113 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:49.241070 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b03f7607-4067-4ba5-a951-174633a0f804" path="/var/lib/kubelet/pods/b03f7607-4067-4ba5-a951-174633a0f804/volumes" Jul 16 21:58:53.339942 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.339908 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t"] Jul 16 21:58:53.340391 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.340365 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b03f7607-4067-4ba5-a951-174633a0f804" containerName="limitador" Jul 16 21:58:53.340391 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.340379 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b03f7607-4067-4ba5-a951-174633a0f804" containerName="limitador" Jul 16 21:58:53.340467 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.340440 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b03f7607-4067-4ba5-a951-174633a0f804" containerName="limitador" Jul 16 21:58:53.346302 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.346283 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:58:53.349529 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.349505 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 16 21:58:53.349641 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.349606 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 16 21:58:53.350303 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.350282 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 16 21:58:53.351063 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.351041 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-x42zb\"" Jul 16 21:58:53.352356 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.352335 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t"] Jul 16 21:58:53.438564 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.438533 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6bml\" (UniqueName: \"kubernetes.io/projected/0c06e1f5-f6e5-47a0-8c1a-fba817fd778e-kube-api-access-z6bml\") pod \"postgres-69fb4594cf-qmg5t\" (UID: \"0c06e1f5-f6e5-47a0-8c1a-fba817fd778e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:58:53.438711 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.438576 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c06e1f5-f6e5-47a0-8c1a-fba817fd778e-data\") pod \"postgres-69fb4594cf-qmg5t\" (UID: \"0c06e1f5-f6e5-47a0-8c1a-fba817fd778e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:58:53.539308 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.539276 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6bml\" (UniqueName: \"kubernetes.io/projected/0c06e1f5-f6e5-47a0-8c1a-fba817fd778e-kube-api-access-z6bml\") pod \"postgres-69fb4594cf-qmg5t\" (UID: \"0c06e1f5-f6e5-47a0-8c1a-fba817fd778e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:58:53.539466 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.539320 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c06e1f5-f6e5-47a0-8c1a-fba817fd778e-data\") pod \"postgres-69fb4594cf-qmg5t\" (UID: \"0c06e1f5-f6e5-47a0-8c1a-fba817fd778e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:58:53.539637 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.539622 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0c06e1f5-f6e5-47a0-8c1a-fba817fd778e-data\") pod \"postgres-69fb4594cf-qmg5t\" (UID: \"0c06e1f5-f6e5-47a0-8c1a-fba817fd778e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:58:53.549180 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.549157 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6bml\" (UniqueName: \"kubernetes.io/projected/0c06e1f5-f6e5-47a0-8c1a-fba817fd778e-kube-api-access-z6bml\") pod \"postgres-69fb4594cf-qmg5t\" (UID: \"0c06e1f5-f6e5-47a0-8c1a-fba817fd778e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:58:53.657606 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.657583 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:58:53.790594 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:53.790567 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t"] Jul 16 21:58:53.791945 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:58:53.791913 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c06e1f5_f6e5_47a0_8c1a_fba817fd778e.slice/crio-63389c335078e8d12bfab6007c73f0a85982e6a5cde11b8e42fb4cc43a0ec7fd WatchSource:0}: Error finding container 63389c335078e8d12bfab6007c73f0a85982e6a5cde11b8e42fb4cc43a0ec7fd: Status 404 returned error can't find the container with id 63389c335078e8d12bfab6007c73f0a85982e6a5cde11b8e42fb4cc43a0ec7fd Jul 16 21:58:54.550891 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:58:54.550855 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" event={"ID":"0c06e1f5-f6e5-47a0-8c1a-fba817fd778e","Type":"ContainerStarted","Data":"63389c335078e8d12bfab6007c73f0a85982e6a5cde11b8e42fb4cc43a0ec7fd"} Jul 16 21:59:00.593096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:00.593007 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" event={"ID":"0c06e1f5-f6e5-47a0-8c1a-fba817fd778e","Type":"ContainerStarted","Data":"11f7342e6a92f13d626a9c65cf256d96d3f17e57f9e6eb1c9950815cc688b486"} Jul 16 21:59:00.593096 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:00.593087 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:59:00.614757 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:00.614707 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" podStartSLOduration=1.169019442 podStartE2EDuration="7.614692012s" totalImagesPullingTime="6.44567257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:59:00.612431355 +0000 UTC m=+735.945510404" watchObservedRunningTime="2026-07-16 21:59:00.614692012 +0000 UTC m=+735.947771042" Jul 16 21:59:06.627803 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:06.627766 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-qmg5t" Jul 16 21:59:07.598119 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.598086 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-99c7dbb9f-pzr9s"] Jul 16 21:59:07.601993 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.601976 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 21:59:07.604918 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.604895 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 16 21:59:07.614731 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.614709 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-99c7dbb9f-pzr9s"] Jul 16 21:59:07.670318 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.670289 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp8dq\" (UniqueName: \"kubernetes.io/projected/b33a449e-08b3-4227-9a5e-67dd72557a80-kube-api-access-zp8dq\") pod \"authorino-99c7dbb9f-pzr9s\" (UID: \"b33a449e-08b3-4227-9a5e-67dd72557a80\") " pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 21:59:07.670318 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.670319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b33a449e-08b3-4227-9a5e-67dd72557a80-tls-cert\") pod \"authorino-99c7dbb9f-pzr9s\" (UID: \"b33a449e-08b3-4227-9a5e-67dd72557a80\") " pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 21:59:07.771632 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.771604 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b33a449e-08b3-4227-9a5e-67dd72557a80-tls-cert\") pod \"authorino-99c7dbb9f-pzr9s\" (UID: \"b33a449e-08b3-4227-9a5e-67dd72557a80\") " pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 21:59:07.771809 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.771705 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp8dq\" (UniqueName: \"kubernetes.io/projected/b33a449e-08b3-4227-9a5e-67dd72557a80-kube-api-access-zp8dq\") pod \"authorino-99c7dbb9f-pzr9s\" (UID: \"b33a449e-08b3-4227-9a5e-67dd72557a80\") " pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 21:59:07.774034 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.774006 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b33a449e-08b3-4227-9a5e-67dd72557a80-tls-cert\") pod \"authorino-99c7dbb9f-pzr9s\" (UID: \"b33a449e-08b3-4227-9a5e-67dd72557a80\") " pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 21:59:07.779544 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.779523 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp8dq\" (UniqueName: \"kubernetes.io/projected/b33a449e-08b3-4227-9a5e-67dd72557a80-kube-api-access-zp8dq\") pod \"authorino-99c7dbb9f-pzr9s\" (UID: \"b33a449e-08b3-4227-9a5e-67dd72557a80\") " pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 21:59:07.910991 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:07.910961 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 21:59:08.044733 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:08.044708 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-99c7dbb9f-pzr9s"] Jul 16 21:59:08.046721 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:59:08.046692 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb33a449e_08b3_4227_9a5e_67dd72557a80.slice/crio-6278f0f09b49a6bb70d18540270306a4ade6933a2fda7879324ff1e3301d8d10 WatchSource:0}: Error finding container 6278f0f09b49a6bb70d18540270306a4ade6933a2fda7879324ff1e3301d8d10: Status 404 returned error can't find the container with id 6278f0f09b49a6bb70d18540270306a4ade6933a2fda7879324ff1e3301d8d10 Jul 16 21:59:08.638027 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:08.638000 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" event={"ID":"b33a449e-08b3-4227-9a5e-67dd72557a80","Type":"ContainerStarted","Data":"6278f0f09b49a6bb70d18540270306a4ade6933a2fda7879324ff1e3301d8d10"} Jul 16 21:59:09.645230 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:09.645195 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" event={"ID":"b33a449e-08b3-4227-9a5e-67dd72557a80","Type":"ContainerStarted","Data":"536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e"} Jul 16 21:59:09.661656 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:09.661608 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" podStartSLOduration=2.107205971 podStartE2EDuration="2.661591973s" totalImagesPullingTime="554.386002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:59:09.660368605 +0000 UTC m=+744.993447635" watchObservedRunningTime="2026-07-16 21:59:09.661591973 +0000 UTC m=+744.994671004" Jul 16 21:59:09.691615 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:09.691580 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hmlzk"] Jul 16 21:59:09.691793 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:09.691770 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hmlzk" podUID="64153789-804f-426a-83fa-765231521d22" containerName="authorino" containerID="cri-o://fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699" gracePeriod=30 Jul 16 21:59:09.949761 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:09.949740 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hmlzk" Jul 16 21:59:09.992428 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:09.992396 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-796vw\" (UniqueName: \"kubernetes.io/projected/64153789-804f-426a-83fa-765231521d22-kube-api-access-796vw\") pod \"64153789-804f-426a-83fa-765231521d22\" (UID: \"64153789-804f-426a-83fa-765231521d22\") " Jul 16 21:59:09.994449 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:09.994414 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64153789-804f-426a-83fa-765231521d22-kube-api-access-796vw" (OuterVolumeSpecName: "kube-api-access-796vw") pod "64153789-804f-426a-83fa-765231521d22" (UID: "64153789-804f-426a-83fa-765231521d22"). InnerVolumeSpecName "kube-api-access-796vw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 21:59:10.093714 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.093687 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-796vw\" (UniqueName: \"kubernetes.io/projected/64153789-804f-426a-83fa-765231521d22-kube-api-access-796vw\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 21:59:10.652230 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.652193 2563 generic.go:356] "Generic (PLEG): container finished" podID="64153789-804f-426a-83fa-765231521d22" containerID="fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699" exitCode=0 Jul 16 21:59:10.652656 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.652236 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hmlzk" Jul 16 21:59:10.652656 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.652281 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hmlzk" event={"ID":"64153789-804f-426a-83fa-765231521d22","Type":"ContainerDied","Data":"fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699"} Jul 16 21:59:10.652656 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.652321 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hmlzk" event={"ID":"64153789-804f-426a-83fa-765231521d22","Type":"ContainerDied","Data":"a19e1c196657c8dc609fc91846046db333e14764530527cee12ae233698a29a3"} Jul 16 21:59:10.652656 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.652336 2563 scope.go:117] "RemoveContainer" containerID="fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699" Jul 16 21:59:10.661511 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.661493 2563 scope.go:117] "RemoveContainer" containerID="fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699" Jul 16 21:59:10.661744 ip-10-0-136-31 kubenswrapper[2563]: E0716 21:59:10.661726 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699\": container with ID starting with fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699 not found: ID does not exist" containerID="fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699" Jul 16 21:59:10.661790 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.661754 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699"} err="failed to get container status \"fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699\": rpc error: code = NotFound desc = could not find container \"fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699\": container with ID starting with fcb42db38ca979a25c81e157800bb719ad92446ea9487323ce863340b90d2699 not found: ID does not exist" Jul 16 21:59:10.677543 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.677507 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hmlzk"] Jul 16 21:59:10.684087 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:10.684066 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hmlzk"] Jul 16 21:59:11.240918 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:11.240881 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64153789-804f-426a-83fa-765231521d22" path="/var/lib/kubelet/pods/64153789-804f-426a-83fa-765231521d22/volumes" Jul 16 21:59:15.059250 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.059221 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-95466d4fd-t6sgt"] Jul 16 21:59:15.059637 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.059624 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64153789-804f-426a-83fa-765231521d22" containerName="authorino" Jul 16 21:59:15.059682 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.059638 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="64153789-804f-426a-83fa-765231521d22" containerName="authorino" Jul 16 21:59:15.059722 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.059706 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="64153789-804f-426a-83fa-765231521d22" containerName="authorino" Jul 16 21:59:15.064188 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.064171 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:15.066802 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.066782 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sckmc\"" Jul 16 21:59:15.066931 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.066785 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 16 21:59:15.068423 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.068402 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 16 21:59:15.072679 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.072344 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95466d4fd-t6sgt"] Jul 16 21:59:15.140249 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.140220 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-webhook-cert\") pod \"maas-controller-95466d4fd-t6sgt\" (UID: \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\") " pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:15.140380 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.140255 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw2xg\" (UniqueName: \"kubernetes.io/projected/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-kube-api-access-sw2xg\") pod \"maas-controller-95466d4fd-t6sgt\" (UID: \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\") " pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:15.241223 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.241198 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-webhook-cert\") pod \"maas-controller-95466d4fd-t6sgt\" (UID: \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\") " pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:15.241362 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.241232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw2xg\" (UniqueName: \"kubernetes.io/projected/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-kube-api-access-sw2xg\") pod \"maas-controller-95466d4fd-t6sgt\" (UID: \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\") " pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:15.243587 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.243558 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-webhook-cert\") pod \"maas-controller-95466d4fd-t6sgt\" (UID: \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\") " pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:15.251702 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.251676 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw2xg\" (UniqueName: \"kubernetes.io/projected/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-kube-api-access-sw2xg\") pod \"maas-controller-95466d4fd-t6sgt\" (UID: \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\") " pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:15.376724 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.376696 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:15.511922 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.511894 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95466d4fd-t6sgt"] Jul 16 21:59:15.512919 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:59:15.512888 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb3187cf_2a4a_4a47_af79_d27c7e95e6b0.slice/crio-50d155e771f8f940ee0e1301b64cc2ca8e3d4b68af57a65530cf1860f5afca92 WatchSource:0}: Error finding container 50d155e771f8f940ee0e1301b64cc2ca8e3d4b68af57a65530cf1860f5afca92: Status 404 returned error can't find the container with id 50d155e771f8f940ee0e1301b64cc2ca8e3d4b68af57a65530cf1860f5afca92 Jul 16 21:59:15.681836 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:15.681759 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95466d4fd-t6sgt" event={"ID":"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0","Type":"ContainerStarted","Data":"50d155e771f8f940ee0e1301b64cc2ca8e3d4b68af57a65530cf1860f5afca92"} Jul 16 21:59:18.703045 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:18.703006 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95466d4fd-t6sgt" event={"ID":"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0","Type":"ContainerStarted","Data":"d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e"} Jul 16 21:59:18.703439 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:18.703164 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:18.725127 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:18.725076 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-95466d4fd-t6sgt" podStartSLOduration=1.322592061 podStartE2EDuration="3.725062371s" totalImagesPullingTime="2.40247031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:59:18.723597618 +0000 UTC m=+754.056676649" watchObservedRunningTime="2026-07-16 21:59:18.725062371 +0000 UTC m=+754.058141401" Jul 16 21:59:29.710234 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:29.710204 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 21:59:50.358301 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.358266 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv"] Jul 16 21:59:50.363528 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.363511 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.367866 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.367836 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 16 21:59:50.367866 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.367837 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-t7zdm\"" Jul 16 21:59:50.373856 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.373834 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv"] Jul 16 21:59:50.447503 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.447467 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84036834-6c09-4d1c-80f9-06c96e1beea9-openshift-service-ca\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.447683 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.447507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84036834-6c09-4d1c-80f9-06c96e1beea9-maas-api-tls\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.447683 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.447602 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrvh7\" (UniqueName: \"kubernetes.io/projected/84036834-6c09-4d1c-80f9-06c96e1beea9-kube-api-access-wrvh7\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.549036 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.549007 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84036834-6c09-4d1c-80f9-06c96e1beea9-openshift-service-ca\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.549212 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.549045 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84036834-6c09-4d1c-80f9-06c96e1beea9-maas-api-tls\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.549212 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.549083 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrvh7\" (UniqueName: \"kubernetes.io/projected/84036834-6c09-4d1c-80f9-06c96e1beea9-kube-api-access-wrvh7\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.549714 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.549690 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84036834-6c09-4d1c-80f9-06c96e1beea9-openshift-service-ca\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.551937 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.551909 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84036834-6c09-4d1c-80f9-06c96e1beea9-maas-api-tls\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.558183 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.558132 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrvh7\" (UniqueName: \"kubernetes.io/projected/84036834-6c09-4d1c-80f9-06c96e1beea9-kube-api-access-wrvh7\") pod \"maas-api-7746cf446f-qkqtv\" (UID: \"84036834-6c09-4d1c-80f9-06c96e1beea9\") " pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.675161 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.675059 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:50.818325 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.818279 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv"] Jul 16 21:59:50.819107 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:59:50.819068 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84036834_6c09_4d1c_80f9_06c96e1beea9.slice/crio-41eaf2d0ee1b4bc84104ac0585705e7459adccf38ecf16c310e9641b55cddaed WatchSource:0}: Error finding container 41eaf2d0ee1b4bc84104ac0585705e7459adccf38ecf16c310e9641b55cddaed: Status 404 returned error can't find the container with id 41eaf2d0ee1b4bc84104ac0585705e7459adccf38ecf16c310e9641b55cddaed Jul 16 21:59:50.880802 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.880771 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" event={"ID":"84036834-6c09-4d1c-80f9-06c96e1beea9","Type":"ContainerStarted","Data":"41eaf2d0ee1b4bc84104ac0585705e7459adccf38ecf16c310e9641b55cddaed"} Jul 16 21:59:50.944450 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.944375 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-pwq2p"] Jul 16 21:59:50.948327 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.948305 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:50.951201 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.951180 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 16 21:59:50.951297 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.951279 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dj5fj\"" Jul 16 21:59:50.958417 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:50.958393 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-pwq2p"] Jul 16 21:59:51.053064 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.053031 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l4st\" (UniqueName: \"kubernetes.io/projected/a333af72-eff1-4360-a8dd-9bfc366bb9ab-kube-api-access-6l4st\") pod \"payload-processing-5bf47cd988-pwq2p\" (UID: \"a333af72-eff1-4360-a8dd-9bfc366bb9ab\") " pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:51.053280 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.053100 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a333af72-eff1-4360-a8dd-9bfc366bb9ab-plugins-config-volume\") pod \"payload-processing-5bf47cd988-pwq2p\" (UID: \"a333af72-eff1-4360-a8dd-9bfc366bb9ab\") " pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:51.154251 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.154212 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a333af72-eff1-4360-a8dd-9bfc366bb9ab-plugins-config-volume\") pod \"payload-processing-5bf47cd988-pwq2p\" (UID: \"a333af72-eff1-4360-a8dd-9bfc366bb9ab\") " pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:51.154432 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.154350 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l4st\" (UniqueName: \"kubernetes.io/projected/a333af72-eff1-4360-a8dd-9bfc366bb9ab-kube-api-access-6l4st\") pod \"payload-processing-5bf47cd988-pwq2p\" (UID: \"a333af72-eff1-4360-a8dd-9bfc366bb9ab\") " pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:51.154976 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.154955 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a333af72-eff1-4360-a8dd-9bfc366bb9ab-plugins-config-volume\") pod \"payload-processing-5bf47cd988-pwq2p\" (UID: \"a333af72-eff1-4360-a8dd-9bfc366bb9ab\") " pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:51.162965 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.162944 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l4st\" (UniqueName: \"kubernetes.io/projected/a333af72-eff1-4360-a8dd-9bfc366bb9ab-kube-api-access-6l4st\") pod \"payload-processing-5bf47cd988-pwq2p\" (UID: \"a333af72-eff1-4360-a8dd-9bfc366bb9ab\") " pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:51.259833 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.259755 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:51.398576 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.398551 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-pwq2p"] Jul 16 21:59:51.400571 ip-10-0-136-31 kubenswrapper[2563]: W0716 21:59:51.400547 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda333af72_eff1_4360_a8dd_9bfc366bb9ab.slice/crio-1b6394803e83ebc1d7a9e8486fd8dcb30082cf037283810ead567f465f533824 WatchSource:0}: Error finding container 1b6394803e83ebc1d7a9e8486fd8dcb30082cf037283810ead567f465f533824: Status 404 returned error can't find the container with id 1b6394803e83ebc1d7a9e8486fd8dcb30082cf037283810ead567f465f533824 Jul 16 21:59:51.889112 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:51.889022 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" event={"ID":"a333af72-eff1-4360-a8dd-9bfc366bb9ab","Type":"ContainerStarted","Data":"1b6394803e83ebc1d7a9e8486fd8dcb30082cf037283810ead567f465f533824"} Jul 16 21:59:52.898009 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:52.897966 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" event={"ID":"84036834-6c09-4d1c-80f9-06c96e1beea9","Type":"ContainerStarted","Data":"753b6d3d0020dfe95fd13c398588bea315f47cc63f564b49433c15971d77f8dd"} Jul 16 21:59:52.898416 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:52.898046 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:52.942098 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:52.942002 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" podStartSLOduration=1.104502034 podStartE2EDuration="2.941985227s" totalImagesPullingTime="1.837483193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:59:52.940417021 +0000 UTC m=+788.273496086" watchObservedRunningTime="2026-07-16 21:59:52.941985227 +0000 UTC m=+788.275064256" Jul 16 21:59:53.905316 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:53.905282 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" event={"ID":"a333af72-eff1-4360-a8dd-9bfc366bb9ab","Type":"ContainerStarted","Data":"30b614e7d8d6e9e15edaa7d36bdd5d6e236ab9069371f285fb26d61c4753c5d2"} Jul 16 21:59:53.905728 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:53.905578 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 21:59:53.925510 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:53.925451 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" podStartSLOduration=1.9335129420000001 podStartE2EDuration="3.925430496s" totalImagesPullingTime="1.991917554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:59:53.921090412 +0000 UTC m=+789.254169445" watchObservedRunningTime="2026-07-16 21:59:53.925430496 +0000 UTC m=+789.258509527" Jul 16 21:59:58.910216 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:58.910185 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7746cf446f-qkqtv" Jul 16 21:59:58.936656 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:58.936624 2563 generic.go:356] "Generic (PLEG): container finished" podID="b3ba87a0-de8b-4043-999d-3204f369eaca" containerID="a7cafce5ede05179bced86dd3039ed0df8a4a759bd4aa18ba0e49b8323889aae" exitCode=0 Jul 16 21:59:58.936948 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:58.936921 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" event={"ID":"b3ba87a0-de8b-4043-999d-3204f369eaca","Type":"ContainerDied","Data":"a7cafce5ede05179bced86dd3039ed0df8a4a759bd4aa18ba0e49b8323889aae"} Jul 16 21:59:58.937337 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:58.937311 2563 scope.go:117] "RemoveContainer" containerID="a7cafce5ede05179bced86dd3039ed0df8a4a759bd4aa18ba0e49b8323889aae" Jul 16 21:59:58.938451 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:58.938421 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 21:59:58.938530 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:58.938494 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 21:59:58.938568 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:58.938533 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 16 21:59:59.949468 ip-10-0-136-31 kubenswrapper[2563]: I0716 21:59:59.949434 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" event={"ID":"b3ba87a0-de8b-4043-999d-3204f369eaca","Type":"ContainerStarted","Data":"ca6e74c450232c0eec3d37d854f6594c6b18a0d54741ccdc93b7bb0383c85524"} Jul 16 22:00:00.848359 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:00.848323 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 22:00:00.849730 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:00.849693 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tvknc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 16 22:00:00.849892 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:00.849762 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" podUID="b3ba87a0-de8b-4043-999d-3204f369eaca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 16 22:00:01.408110 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:01.408067 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 22:00:01.848653 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:01.848558 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tvknc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 16 22:00:01.848653 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:01.848628 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" podUID="b3ba87a0-de8b-4043-999d-3204f369eaca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 16 22:00:02.848658 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:02.848617 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tvknc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 16 22:00:02.849044 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:02.848682 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" podUID="b3ba87a0-de8b-4043-999d-3204f369eaca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 16 22:00:03.848613 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:03.848570 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tvknc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 16 22:00:03.848799 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:03.848642 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" podUID="b3ba87a0-de8b-4043-999d-3204f369eaca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 16 22:00:04.852970 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:04.852941 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 22:00:04.853995 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:04.853973 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tvknc" Jul 16 22:00:04.913185 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:04.913136 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-pwq2p" Jul 16 22:00:13.216071 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.215988 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl"] Jul 16 22:00:13.219637 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.219615 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.222825 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.222794 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 22:00:13.223037 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.223022 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 22:00:13.224045 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.224027 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rr6sx\"" Jul 16 22:00:13.224155 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.224062 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 16 22:00:13.231021 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.230994 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl"] Jul 16 22:00:13.372603 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.372563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.372777 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.372623 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rsls\" (UniqueName: \"kubernetes.io/projected/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-kube-api-access-2rsls\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.372777 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.372673 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.372777 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.372703 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.372930 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.372802 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.372930 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.372852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474087 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474000 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474087 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474076 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rsls\" (UniqueName: \"kubernetes.io/projected/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-kube-api-access-2rsls\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474346 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474346 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474198 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474346 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474248 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474346 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474631 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474606 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474631 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474621 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.474733 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.474659 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.476169 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.476134 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.476526 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.476509 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.482671 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.482652 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rsls\" (UniqueName: \"kubernetes.io/projected/f32db882-0d9a-43ec-b7a2-8e6ba92bf849-kube-api-access-2rsls\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl\" (UID: \"f32db882-0d9a-43ec-b7a2-8e6ba92bf849\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.532933 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.532905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:13.700733 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:13.700697 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl"] Jul 16 22:00:13.704726 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:00:13.704700 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf32db882_0d9a_43ec_b7a2_8e6ba92bf849.slice/crio-7ca746ae97c0b9fe37fc19650bc5d667314fd4836d1ccd86ef5fe282105718e9 WatchSource:0}: Error finding container 7ca746ae97c0b9fe37fc19650bc5d667314fd4836d1ccd86ef5fe282105718e9: Status 404 returned error can't find the container with id 7ca746ae97c0b9fe37fc19650bc5d667314fd4836d1ccd86ef5fe282105718e9 Jul 16 22:00:14.040360 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:14.040326 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" event={"ID":"f32db882-0d9a-43ec-b7a2-8e6ba92bf849","Type":"ContainerStarted","Data":"7ca746ae97c0b9fe37fc19650bc5d667314fd4836d1ccd86ef5fe282105718e9"} Jul 16 22:00:38.191168 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.191106 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" event={"ID":"f32db882-0d9a-43ec-b7a2-8e6ba92bf849","Type":"ContainerStarted","Data":"0ecfdaaaff7070559d35f627c00a3d17d2bd0bd183ff2da5a294c924118ae05b"} Jul 16 22:00:38.211195 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.211126 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" podStartSLOduration=1.65140548 podStartE2EDuration="25.211110235s" totalImagesPullingTime="23.559704755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:00:38.210579792 +0000 UTC m=+833.543658824" watchObservedRunningTime="2026-07-16 22:00:38.211110235 +0000 UTC m=+833.544189267" Jul 16 22:00:38.604357 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.604319 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q"] Jul 16 22:00:38.606985 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.606964 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.609951 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.609925 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 16 22:00:38.619099 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.619077 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q"] Jul 16 22:00:38.716259 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.716222 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3876ed7-b133-4bdb-a5e7-00b13330fc74-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.716441 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.716268 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qngpk\" (UniqueName: \"kubernetes.io/projected/b3876ed7-b133-4bdb-a5e7-00b13330fc74-kube-api-access-qngpk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.716441 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.716330 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.716441 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.716385 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.716612 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.716498 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.716612 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.716572 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.817938 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.817900 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.818127 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.817955 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3876ed7-b133-4bdb-a5e7-00b13330fc74-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.818127 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.818082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qngpk\" (UniqueName: \"kubernetes.io/projected/b3876ed7-b133-4bdb-a5e7-00b13330fc74-kube-api-access-qngpk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.818300 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.818128 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.818300 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.818182 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.818410 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.818303 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.818410 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.818313 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.818563 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.818542 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.818628 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.818613 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.820241 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.820219 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3876ed7-b133-4bdb-a5e7-00b13330fc74-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.820513 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.820494 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3876ed7-b133-4bdb-a5e7-00b13330fc74-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.826849 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.826823 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qngpk\" (UniqueName: \"kubernetes.io/projected/b3876ed7-b133-4bdb-a5e7-00b13330fc74-kube-api-access-qngpk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-whp4q\" (UID: \"b3876ed7-b133-4bdb-a5e7-00b13330fc74\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:38.918789 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:38.918712 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:39.056652 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:39.056625 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q"] Jul 16 22:00:39.057441 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:00:39.057420 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3876ed7_b133_4bdb_a5e7_00b13330fc74.slice/crio-c0ea763a532445f6f30e6e907999300e0fdd48be4143ff5debd1953fc920d2e0 WatchSource:0}: Error finding container c0ea763a532445f6f30e6e907999300e0fdd48be4143ff5debd1953fc920d2e0: Status 404 returned error can't find the container with id c0ea763a532445f6f30e6e907999300e0fdd48be4143ff5debd1953fc920d2e0 Jul 16 22:00:39.198591 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:39.198505 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" event={"ID":"b3876ed7-b133-4bdb-a5e7-00b13330fc74","Type":"ContainerStarted","Data":"c0ea763a532445f6f30e6e907999300e0fdd48be4143ff5debd1953fc920d2e0"} Jul 16 22:00:43.533786 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:43.533749 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:43.534205 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:43.533821 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:43.553812 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:43.553781 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:44.239929 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:44.239897 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl" Jul 16 22:00:51.271862 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:51.271826 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" event={"ID":"b3876ed7-b133-4bdb-a5e7-00b13330fc74","Type":"ContainerStarted","Data":"87876e0a92e685bf8c7c591e5238dd31bd39c248d3d0584343b606e0068c88ad"} Jul 16 22:00:51.293722 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:51.293674 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" podStartSLOduration=1.883279725 podStartE2EDuration="13.293658899s" totalImagesPullingTime="11.410379174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:00:51.290543939 +0000 UTC m=+846.623622969" watchObservedRunningTime="2026-07-16 22:00:51.293658899 +0000 UTC m=+846.626737928" Jul 16 22:00:58.918882 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:58.918847 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:58.919387 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:58.918904 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:58.930996 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:58.930974 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:00:59.325876 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:00:59.325795 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-whp4q" Jul 16 22:01:09.518324 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.518288 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc"] Jul 16 22:01:09.523042 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.523026 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.530169 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.530121 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 16 22:01:09.540617 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.540594 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc"] Jul 16 22:01:09.610180 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.610113 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.610180 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.610181 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.610373 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.610300 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb882\" (UniqueName: \"kubernetes.io/projected/acabb8ac-2574-4f31-bcb3-1d2c99423669-kube-api-access-nb882\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.610373 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.610339 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.610373 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.610363 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acabb8ac-2574-4f31-bcb3-1d2c99423669-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.610535 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.610494 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.711431 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.711401 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.711608 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.711447 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.711608 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.711465 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.711608 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.711508 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb882\" (UniqueName: \"kubernetes.io/projected/acabb8ac-2574-4f31-bcb3-1d2c99423669-kube-api-access-nb882\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.711841 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.711801 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.711933 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.711845 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.711933 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.711862 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acabb8ac-2574-4f31-bcb3-1d2c99423669-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.711933 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.711900 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.712247 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.712225 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.713887 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.713865 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acabb8ac-2574-4f31-bcb3-1d2c99423669-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.714353 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.714335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acabb8ac-2574-4f31-bcb3-1d2c99423669-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.728021 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.727999 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb882\" (UniqueName: \"kubernetes.io/projected/acabb8ac-2574-4f31-bcb3-1d2c99423669-kube-api-access-nb882\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc\" (UID: \"acabb8ac-2574-4f31-bcb3-1d2c99423669\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.832718 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.832649 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:09.973257 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:09.973231 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc"] Jul 16 22:01:09.973645 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:01:09.973620 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacabb8ac_2574_4f31_bcb3_1d2c99423669.slice/crio-badf28d361894b8c72500013bb4f61c756615a287b51b988c1778d45f86a20c6 WatchSource:0}: Error finding container badf28d361894b8c72500013bb4f61c756615a287b51b988c1778d45f86a20c6: Status 404 returned error can't find the container with id badf28d361894b8c72500013bb4f61c756615a287b51b988c1778d45f86a20c6 Jul 16 22:01:10.381312 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:10.381280 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" event={"ID":"acabb8ac-2574-4f31-bcb3-1d2c99423669","Type":"ContainerStarted","Data":"badf28d361894b8c72500013bb4f61c756615a287b51b988c1778d45f86a20c6"} Jul 16 22:01:13.401517 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:13.401481 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" event={"ID":"acabb8ac-2574-4f31-bcb3-1d2c99423669","Type":"ContainerStarted","Data":"66f07207cef53006170db30321b83e0b702237bccfdc48a68f2b89fdfb73cedf"} Jul 16 22:01:13.422269 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:13.422224 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" podStartSLOduration=1.92949747 podStartE2EDuration="4.422209289s" totalImagesPullingTime="2.492711819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:01:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:01:13.41928888 +0000 UTC m=+868.752367910" watchObservedRunningTime="2026-07-16 22:01:13.422209289 +0000 UTC m=+868.755288360" Jul 16 22:01:19.833400 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:19.833360 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:19.833834 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:19.833431 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:19.845566 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:19.845541 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:20.453574 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:20.453545 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc" Jul 16 22:01:45.262321 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:45.262286 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:01:45.264307 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:45.264289 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:01:45.265451 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:45.265431 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:01:45.267014 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:01:45.266996 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:02:28.900354 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:28.900321 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-578d6d868b-pcf2f"] Jul 16 22:02:28.904162 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:28.904132 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-578d6d868b-pcf2f" Jul 16 22:02:28.929376 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:28.929353 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-578d6d868b-pcf2f"] Jul 16 22:02:28.956287 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:28.956263 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4-tls-cert\") pod \"authorino-578d6d868b-pcf2f\" (UID: \"ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4\") " pod="kuadrant-system/authorino-578d6d868b-pcf2f" Jul 16 22:02:28.956406 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:28.956332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz8g8\" (UniqueName: \"kubernetes.io/projected/ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4-kube-api-access-jz8g8\") pod \"authorino-578d6d868b-pcf2f\" (UID: \"ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4\") " pod="kuadrant-system/authorino-578d6d868b-pcf2f" Jul 16 22:02:29.057690 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:29.057657 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz8g8\" (UniqueName: \"kubernetes.io/projected/ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4-kube-api-access-jz8g8\") pod \"authorino-578d6d868b-pcf2f\" (UID: \"ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4\") " pod="kuadrant-system/authorino-578d6d868b-pcf2f" Jul 16 22:02:29.057870 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:29.057776 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4-tls-cert\") pod \"authorino-578d6d868b-pcf2f\" (UID: \"ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4\") " pod="kuadrant-system/authorino-578d6d868b-pcf2f" Jul 16 22:02:29.060212 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:29.060180 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4-tls-cert\") pod \"authorino-578d6d868b-pcf2f\" (UID: \"ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4\") " pod="kuadrant-system/authorino-578d6d868b-pcf2f" Jul 16 22:02:29.066768 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:29.066746 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz8g8\" (UniqueName: \"kubernetes.io/projected/ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4-kube-api-access-jz8g8\") pod \"authorino-578d6d868b-pcf2f\" (UID: \"ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4\") " pod="kuadrant-system/authorino-578d6d868b-pcf2f" Jul 16 22:02:29.213025 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:29.212939 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-578d6d868b-pcf2f" Jul 16 22:02:29.355956 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:29.355925 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-578d6d868b-pcf2f"] Jul 16 22:02:29.362257 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:29.362234 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 22:02:29.854487 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:29.854456 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-578d6d868b-pcf2f" event={"ID":"ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4","Type":"ContainerStarted","Data":"034fa9c778c913635654b2dfe0f4d21b98faa2f58dc9b907ca8f0da2f7c7c7a1"} Jul 16 22:02:30.864530 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:30.864493 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-578d6d868b-pcf2f" event={"ID":"ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4","Type":"ContainerStarted","Data":"e689076779100ed60436135b1a899650059c0aa2716cc75313e84bbf4a75bcef"} Jul 16 22:02:30.887403 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:30.887326 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-578d6d868b-pcf2f" podStartSLOduration=2.472780764 podStartE2EDuration="2.887304905s" totalImagesPullingTime="414.524141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:02:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:02:30.883086836 +0000 UTC m=+946.216165869" watchObservedRunningTime="2026-07-16 22:02:30.887304905 +0000 UTC m=+946.220383936" Jul 16 22:02:30.921163 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:30.920400 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-99c7dbb9f-pzr9s"] Jul 16 22:02:30.921163 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:30.920658 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" podUID="b33a449e-08b3-4227-9a5e-67dd72557a80" containerName="authorino" containerID="cri-o://536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e" gracePeriod=30 Jul 16 22:02:31.176732 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.176699 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 22:02:31.177634 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.177609 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zp8dq\" (UniqueName: \"kubernetes.io/projected/b33a449e-08b3-4227-9a5e-67dd72557a80-kube-api-access-zp8dq\") pod \"b33a449e-08b3-4227-9a5e-67dd72557a80\" (UID: \"b33a449e-08b3-4227-9a5e-67dd72557a80\") " Jul 16 22:02:31.177772 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.177653 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b33a449e-08b3-4227-9a5e-67dd72557a80-tls-cert\") pod \"b33a449e-08b3-4227-9a5e-67dd72557a80\" (UID: \"b33a449e-08b3-4227-9a5e-67dd72557a80\") " Jul 16 22:02:31.179659 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.179633 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b33a449e-08b3-4227-9a5e-67dd72557a80-kube-api-access-zp8dq" (OuterVolumeSpecName: "kube-api-access-zp8dq") pod "b33a449e-08b3-4227-9a5e-67dd72557a80" (UID: "b33a449e-08b3-4227-9a5e-67dd72557a80"). InnerVolumeSpecName "kube-api-access-zp8dq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:02:31.192273 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.192245 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b33a449e-08b3-4227-9a5e-67dd72557a80-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b33a449e-08b3-4227-9a5e-67dd72557a80" (UID: "b33a449e-08b3-4227-9a5e-67dd72557a80"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:02:31.279242 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.279202 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp8dq\" (UniqueName: \"kubernetes.io/projected/b33a449e-08b3-4227-9a5e-67dd72557a80-kube-api-access-zp8dq\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:02:31.279242 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.279227 2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b33a449e-08b3-4227-9a5e-67dd72557a80-tls-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:02:31.872898 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.872865 2563 generic.go:356] "Generic (PLEG): container finished" podID="b33a449e-08b3-4227-9a5e-67dd72557a80" containerID="536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e" exitCode=0 Jul 16 22:02:31.873423 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.872914 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" Jul 16 22:02:31.873423 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.872951 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" event={"ID":"b33a449e-08b3-4227-9a5e-67dd72557a80","Type":"ContainerDied","Data":"536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e"} Jul 16 22:02:31.873423 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.872993 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99c7dbb9f-pzr9s" event={"ID":"b33a449e-08b3-4227-9a5e-67dd72557a80","Type":"ContainerDied","Data":"6278f0f09b49a6bb70d18540270306a4ade6933a2fda7879324ff1e3301d8d10"} Jul 16 22:02:31.873423 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.873012 2563 scope.go:117] "RemoveContainer" containerID="536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e" Jul 16 22:02:31.883468 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.883443 2563 scope.go:117] "RemoveContainer" containerID="536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e" Jul 16 22:02:31.883779 ip-10-0-136-31 kubenswrapper[2563]: E0716 22:02:31.883759 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e\": container with ID starting with 536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e not found: ID does not exist" containerID="536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e" Jul 16 22:02:31.883850 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.883788 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e"} err="failed to get container status \"536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e\": rpc error: code = NotFound desc = could not find container \"536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e\": container with ID starting with 536ce3db5319a40f3f15a502cd857727db08f83b49c5bf1443eb77386cfa711e not found: ID does not exist" Jul 16 22:02:31.892712 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.892690 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-99c7dbb9f-pzr9s"] Jul 16 22:02:31.910484 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:31.910461 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-99c7dbb9f-pzr9s"] Jul 16 22:02:33.241878 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:02:33.241844 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b33a449e-08b3-4227-9a5e-67dd72557a80" path="/var/lib/kubelet/pods/b33a449e-08b3-4227-9a5e-67dd72557a80/volumes" Jul 16 22:03:18.890325 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:18.890234 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-95466d4fd-t6sgt"] Jul 16 22:03:18.890887 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:18.890513 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-95466d4fd-t6sgt" podUID="eb3187cf-2a4a-4a47-af79-d27c7e95e6b0" containerName="manager" containerID="cri-o://d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e" gracePeriod=30 Jul 16 22:03:19.143363 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.143305 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 22:03:19.165762 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.165733 2563 generic.go:356] "Generic (PLEG): container finished" podID="eb3187cf-2a4a-4a47-af79-d27c7e95e6b0" containerID="d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e" exitCode=0 Jul 16 22:03:19.165915 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.165777 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95466d4fd-t6sgt" Jul 16 22:03:19.165915 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.165815 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95466d4fd-t6sgt" event={"ID":"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0","Type":"ContainerDied","Data":"d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e"} Jul 16 22:03:19.165915 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.165867 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95466d4fd-t6sgt" event={"ID":"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0","Type":"ContainerDied","Data":"50d155e771f8f940ee0e1301b64cc2ca8e3d4b68af57a65530cf1860f5afca92"} Jul 16 22:03:19.165915 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.165890 2563 scope.go:117] "RemoveContainer" containerID="d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e" Jul 16 22:03:19.178280 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.178254 2563 scope.go:117] "RemoveContainer" containerID="d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e" Jul 16 22:03:19.178636 ip-10-0-136-31 kubenswrapper[2563]: E0716 22:03:19.178607 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e\": container with ID starting with d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e not found: ID does not exist" containerID="d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e" Jul 16 22:03:19.178715 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.178651 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e"} err="failed to get container status \"d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e\": rpc error: code = NotFound desc = could not find container \"d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e\": container with ID starting with d3d9fe1a279fb869c5d1bd07184ece90cbebcf5ecf71ded9deb3b877d83ae37e not found: ID does not exist" Jul 16 22:03:19.241527 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.241497 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-webhook-cert\") pod \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\" (UID: \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\") " Jul 16 22:03:19.241674 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.241560 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw2xg\" (UniqueName: \"kubernetes.io/projected/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-kube-api-access-sw2xg\") pod \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\" (UID: \"eb3187cf-2a4a-4a47-af79-d27c7e95e6b0\") " Jul 16 22:03:19.243753 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.243726 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eb3187cf-2a4a-4a47-af79-d27c7e95e6b0" (UID: "eb3187cf-2a4a-4a47-af79-d27c7e95e6b0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 22:03:19.243855 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.243755 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-kube-api-access-sw2xg" (OuterVolumeSpecName: "kube-api-access-sw2xg") pod "eb3187cf-2a4a-4a47-af79-d27c7e95e6b0" (UID: "eb3187cf-2a4a-4a47-af79-d27c7e95e6b0"). InnerVolumeSpecName "kube-api-access-sw2xg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:03:19.343413 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.343390 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-webhook-cert\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:03:19.343413 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.343412 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw2xg\" (UniqueName: \"kubernetes.io/projected/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0-kube-api-access-sw2xg\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:03:19.493911 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.493883 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-95466d4fd-t6sgt"] Jul 16 22:03:19.499522 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:19.499483 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-95466d4fd-t6sgt"] Jul 16 22:03:21.249582 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:03:21.249547 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb3187cf-2a4a-4a47-af79-d27c7e95e6b0" path="/var/lib/kubelet/pods/eb3187cf-2a4a-4a47-af79-d27c7e95e6b0/volumes" Jul 16 22:06:45.307688 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:06:45.307661 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:06:45.310860 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:06:45.310835 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:06:45.311310 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:06:45.311293 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:06:45.314639 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:06:45.314621 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:11:45.348543 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:11:45.348513 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:11:45.351515 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:11:45.351495 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:11:45.355282 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:11:45.355264 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:11:45.358510 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:11:45.358493 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:16:45.389911 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:16:45.389874 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:16:45.393084 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:16:45.393061 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:16:45.398760 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:16:45.398741 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:16:45.401792 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:16:45.401776 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:21:45.432358 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:21:45.432332 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:21:45.435602 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:21:45.435579 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:21:45.444762 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:21:45.444743 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:21:45.447946 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:21:45.447929 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:26:45.472423 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:26:45.472386 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:26:45.475475 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:26:45.475449 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:26:45.486875 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:26:45.486856 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:26:45.489907 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:26:45.489888 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:31:45.516694 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:31:45.516581 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:31:45.519930 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:31:45.519909 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:31:45.535455 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:31:45.535435 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:31:45.538473 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:31:45.538459 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:36:45.556808 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:36:45.556695 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:36:45.561117 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:36:45.560167 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:36:45.578249 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:36:45.578228 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:36:45.581374 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:36:45.581357 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:41:45.597777 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:45.597670 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:41:45.601933 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:45.601237 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:41:45.620266 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:45.620246 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:41:45.623229 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:45.623214 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:41:46.641256 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.641226 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jzpp"] Jul 16 22:41:46.641683 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.641670 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b33a449e-08b3-4227-9a5e-67dd72557a80" containerName="authorino" Jul 16 22:41:46.641683 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.641684 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b33a449e-08b3-4227-9a5e-67dd72557a80" containerName="authorino" Jul 16 22:41:46.641769 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.641692 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb3187cf-2a4a-4a47-af79-d27c7e95e6b0" containerName="manager" Jul 16 22:41:46.641769 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.641699 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb3187cf-2a4a-4a47-af79-d27c7e95e6b0" containerName="manager" Jul 16 22:41:46.641835 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.641796 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb3187cf-2a4a-4a47-af79-d27c7e95e6b0" containerName="manager" Jul 16 22:41:46.641835 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.641808 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b33a449e-08b3-4227-9a5e-67dd72557a80" containerName="authorino" Jul 16 22:41:46.644857 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.644841 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:46.656604 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.656576 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jzpp"] Jul 16 22:41:46.710253 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.710221 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4zps\" (UniqueName: \"kubernetes.io/projected/254954b5-0d35-4c5c-93d2-f7db594148c7-kube-api-access-t4zps\") pod \"kuadrant-operator-catalog-7jzpp\" (UID: \"254954b5-0d35-4c5c-93d2-f7db594148c7\") " pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:46.811254 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.811226 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4zps\" (UniqueName: \"kubernetes.io/projected/254954b5-0d35-4c5c-93d2-f7db594148c7-kube-api-access-t4zps\") pod \"kuadrant-operator-catalog-7jzpp\" (UID: \"254954b5-0d35-4c5c-93d2-f7db594148c7\") " pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:46.824998 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.824974 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4zps\" (UniqueName: \"kubernetes.io/projected/254954b5-0d35-4c5c-93d2-f7db594148c7-kube-api-access-t4zps\") pod \"kuadrant-operator-catalog-7jzpp\" (UID: \"254954b5-0d35-4c5c-93d2-f7db594148c7\") " pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:46.955215 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:46.955126 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:47.109269 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:47.109235 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jzpp"] Jul 16 22:41:47.114126 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:47.114098 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 22:41:47.789381 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:47.789342 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" event={"ID":"254954b5-0d35-4c5c-93d2-f7db594148c7","Type":"ContainerStarted","Data":"fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a"} Jul 16 22:41:47.789763 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:47.789398 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" event={"ID":"254954b5-0d35-4c5c-93d2-f7db594148c7","Type":"ContainerStarted","Data":"93192af67bd50fb9c251a266b42ce8d76ff176e44a3714d02b24775b8b7fcacc"} Jul 16 22:41:47.818173 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:47.818119 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" podStartSLOduration=1.36018795 podStartE2EDuration="1.818103991s" totalImagesPullingTime="457.916041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 22:41:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:41:47.81769126 +0000 UTC m=+3303.150770312" watchObservedRunningTime="2026-07-16 22:41:47.818103991 +0000 UTC m=+3303.151183021" Jul 16 22:41:56.955791 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:56.955756 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:56.956243 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:56.955813 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:56.978393 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:56.978368 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:57.870065 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:57.870040 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:41:57.925203 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:57.925166 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jzpp"] Jul 16 22:41:59.858933 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:41:59.858878 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" podUID="254954b5-0d35-4c5c-93d2-f7db594148c7" containerName="registry-server" containerID="cri-o://fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a" gracePeriod=2 Jul 16 22:42:00.116180 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.116107 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:42:00.239593 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.239560 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4zps\" (UniqueName: \"kubernetes.io/projected/254954b5-0d35-4c5c-93d2-f7db594148c7-kube-api-access-t4zps\") pod \"254954b5-0d35-4c5c-93d2-f7db594148c7\" (UID: \"254954b5-0d35-4c5c-93d2-f7db594148c7\") " Jul 16 22:42:00.241626 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.241591 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/254954b5-0d35-4c5c-93d2-f7db594148c7-kube-api-access-t4zps" (OuterVolumeSpecName: "kube-api-access-t4zps") pod "254954b5-0d35-4c5c-93d2-f7db594148c7" (UID: "254954b5-0d35-4c5c-93d2-f7db594148c7"). InnerVolumeSpecName "kube-api-access-t4zps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:42:00.341192 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.341131 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4zps\" (UniqueName: \"kubernetes.io/projected/254954b5-0d35-4c5c-93d2-f7db594148c7-kube-api-access-t4zps\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:42:00.865505 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.865466 2563 generic.go:356] "Generic (PLEG): container finished" podID="254954b5-0d35-4c5c-93d2-f7db594148c7" containerID="fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a" exitCode=0 Jul 16 22:42:00.865917 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.865513 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" Jul 16 22:42:00.865917 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.865553 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" event={"ID":"254954b5-0d35-4c5c-93d2-f7db594148c7","Type":"ContainerDied","Data":"fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a"} Jul 16 22:42:00.865917 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.865608 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7jzpp" event={"ID":"254954b5-0d35-4c5c-93d2-f7db594148c7","Type":"ContainerDied","Data":"93192af67bd50fb9c251a266b42ce8d76ff176e44a3714d02b24775b8b7fcacc"} Jul 16 22:42:00.865917 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.865629 2563 scope.go:117] "RemoveContainer" containerID="fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a" Jul 16 22:42:00.875740 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.875721 2563 scope.go:117] "RemoveContainer" containerID="fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a" Jul 16 22:42:00.875977 ip-10-0-136-31 kubenswrapper[2563]: E0716 22:42:00.875961 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a\": container with ID starting with fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a not found: ID does not exist" containerID="fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a" Jul 16 22:42:00.876028 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.875987 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a"} err="failed to get container status \"fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a\": rpc error: code = NotFound desc = could not find container \"fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a\": container with ID starting with fa47d48698e8be9d3c8a97fba955ce58b5ab399c736cfc51d9beed03b708b98a not found: ID does not exist" Jul 16 22:42:00.889604 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.889580 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jzpp"] Jul 16 22:42:00.894836 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:00.894816 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7jzpp"] Jul 16 22:42:01.244845 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:42:01.244771 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="254954b5-0d35-4c5c-93d2-f7db594148c7" path="/var/lib/kubelet/pods/254954b5-0d35-4c5c-93d2-f7db594148c7/volumes" Jul 16 22:46:45.637896 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:46:45.637867 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:46:45.641316 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:46:45.641292 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:46:45.662536 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:46:45.662516 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:46:45.665601 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:46:45.665586 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7xmnr_1fd22b1e-b443-499f-b9b0-25bdb51e42e5/ovn-acl-logging/0.log" Jul 16 22:47:24.661414 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.661375 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7194-140373846210864"] Jul 16 22:47:24.661879 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.661851 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="254954b5-0d35-4c5c-93d2-f7db594148c7" containerName="registry-server" Jul 16 22:47:24.661879 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.661864 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="254954b5-0d35-4c5c-93d2-f7db594148c7" containerName="registry-server" Jul 16 22:47:24.661966 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.661951 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="254954b5-0d35-4c5c-93d2-f7db594148c7" containerName="registry-server" Jul 16 22:47:24.665107 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.665092 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846210864" Jul 16 22:47:24.668463 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.668430 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-9zmpg\"" Jul 16 22:47:24.671736 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.671709 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846210864"] Jul 16 22:47:24.765542 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.765513 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtrpp\" (UniqueName: \"kubernetes.io/projected/ad78efb4-7215-4498-bcf3-dd978aaf4605-kube-api-access-xtrpp\") pod \"test-curl-7194-140373846210864\" (UID: \"ad78efb4-7215-4498-bcf3-dd978aaf4605\") " pod="opendatahub/test-curl-7194-140373846210864" Jul 16 22:47:24.866548 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.866507 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtrpp\" (UniqueName: \"kubernetes.io/projected/ad78efb4-7215-4498-bcf3-dd978aaf4605-kube-api-access-xtrpp\") pod \"test-curl-7194-140373846210864\" (UID: \"ad78efb4-7215-4498-bcf3-dd978aaf4605\") " pod="opendatahub/test-curl-7194-140373846210864" Jul 16 22:47:24.874792 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.874769 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtrpp\" (UniqueName: \"kubernetes.io/projected/ad78efb4-7215-4498-bcf3-dd978aaf4605-kube-api-access-xtrpp\") pod \"test-curl-7194-140373846210864\" (UID: \"ad78efb4-7215-4498-bcf3-dd978aaf4605\") " pod="opendatahub/test-curl-7194-140373846210864" Jul 16 22:47:24.976181 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:24.976082 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846210864" Jul 16 22:47:25.111709 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:25.111681 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846210864"] Jul 16 22:47:25.113110 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:47:25.113084 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad78efb4_7215_4498_bcf3_dd978aaf4605.slice/crio-cdf0fae3b71d7fa4dbc28476d9e22d8b26e3bf1ebfc9926783275537e51e5c39 WatchSource:0}: Error finding container cdf0fae3b71d7fa4dbc28476d9e22d8b26e3bf1ebfc9926783275537e51e5c39: Status 404 returned error can't find the container with id cdf0fae3b71d7fa4dbc28476d9e22d8b26e3bf1ebfc9926783275537e51e5c39 Jul 16 22:47:25.114623 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:25.114604 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 22:47:25.772030 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:25.771998 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846210864" event={"ID":"ad78efb4-7215-4498-bcf3-dd978aaf4605","Type":"ContainerStarted","Data":"cdf0fae3b71d7fa4dbc28476d9e22d8b26e3bf1ebfc9926783275537e51e5c39"} Jul 16 22:47:26.778925 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:26.778892 2563 generic.go:356] "Generic (PLEG): container finished" podID="ad78efb4-7215-4498-bcf3-dd978aaf4605" containerID="e13720f70ff64014f885e9fabbbcd072678a1fd3a41f56e43601c43b83a5bb4f" exitCode=0 Jul 16 22:47:26.778925 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:26.778928 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846210864" event={"ID":"ad78efb4-7215-4498-bcf3-dd978aaf4605","Type":"ContainerDied","Data":"e13720f70ff64014f885e9fabbbcd072678a1fd3a41f56e43601c43b83a5bb4f"} Jul 16 22:47:27.930442 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:27.930416 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846210864" Jul 16 22:47:27.968730 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:27.968702 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7194-140373846210864_ad78efb4-7215-4498-bcf3-dd978aaf4605/test-curl-7194-140373846210864/0.log" Jul 16 22:47:27.994250 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:27.994216 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7194-140373846210864"] Jul 16 22:47:27.996851 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:27.996829 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtrpp\" (UniqueName: \"kubernetes.io/projected/ad78efb4-7215-4498-bcf3-dd978aaf4605-kube-api-access-xtrpp\") pod \"ad78efb4-7215-4498-bcf3-dd978aaf4605\" (UID: \"ad78efb4-7215-4498-bcf3-dd978aaf4605\") " Jul 16 22:47:27.999049 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:27.999017 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad78efb4-7215-4498-bcf3-dd978aaf4605-kube-api-access-xtrpp" (OuterVolumeSpecName: "kube-api-access-xtrpp") pod "ad78efb4-7215-4498-bcf3-dd978aaf4605" (UID: "ad78efb4-7215-4498-bcf3-dd978aaf4605"). InnerVolumeSpecName "kube-api-access-xtrpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:47:28.002521 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.002498 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7194-140373846210864"] Jul 16 22:47:28.097961 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.097866 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtrpp\" (UniqueName: \"kubernetes.io/projected/ad78efb4-7215-4498-bcf3-dd978aaf4605-kube-api-access-xtrpp\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:47:28.128904 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.128871 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7194-140373846210992"] Jul 16 22:47:28.129326 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.129312 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad78efb4-7215-4498-bcf3-dd978aaf4605" containerName="test-curl-7194-140373846210864" Jul 16 22:47:28.129376 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.129328 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad78efb4-7215-4498-bcf3-dd978aaf4605" containerName="test-curl-7194-140373846210864" Jul 16 22:47:28.129462 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.129404 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad78efb4-7215-4498-bcf3-dd978aaf4605" containerName="test-curl-7194-140373846210864" Jul 16 22:47:28.132466 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.132450 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846210992" Jul 16 22:47:28.141487 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.141465 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846210992"] Jul 16 22:47:28.198760 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.198713 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf56b\" (UniqueName: \"kubernetes.io/projected/580506e5-2d02-4561-96eb-19619e13ee1b-kube-api-access-zf56b\") pod \"test-curl-7194-140373846210992\" (UID: \"580506e5-2d02-4561-96eb-19619e13ee1b\") " pod="opendatahub/test-curl-7194-140373846210992" Jul 16 22:47:28.299326 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.299283 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf56b\" (UniqueName: \"kubernetes.io/projected/580506e5-2d02-4561-96eb-19619e13ee1b-kube-api-access-zf56b\") pod \"test-curl-7194-140373846210992\" (UID: \"580506e5-2d02-4561-96eb-19619e13ee1b\") " pod="opendatahub/test-curl-7194-140373846210992" Jul 16 22:47:28.309715 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.309685 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf56b\" (UniqueName: \"kubernetes.io/projected/580506e5-2d02-4561-96eb-19619e13ee1b-kube-api-access-zf56b\") pod \"test-curl-7194-140373846210992\" (UID: \"580506e5-2d02-4561-96eb-19619e13ee1b\") " pod="opendatahub/test-curl-7194-140373846210992" Jul 16 22:47:28.443113 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.443078 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846210992" Jul 16 22:47:28.587385 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.587349 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846210992"] Jul 16 22:47:28.588910 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:47:28.588883 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod580506e5_2d02_4561_96eb_19619e13ee1b.slice/crio-db0f2ec4492c59e64032d5f67c69bf5db057ed9cab859cb516856cbbd7dc6d3c WatchSource:0}: Error finding container db0f2ec4492c59e64032d5f67c69bf5db057ed9cab859cb516856cbbd7dc6d3c: Status 404 returned error can't find the container with id db0f2ec4492c59e64032d5f67c69bf5db057ed9cab859cb516856cbbd7dc6d3c Jul 16 22:47:28.791741 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.791652 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846210864" Jul 16 22:47:28.791741 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.791695 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdf0fae3b71d7fa4dbc28476d9e22d8b26e3bf1ebfc9926783275537e51e5c39" Jul 16 22:47:28.793007 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:28.792977 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846210992" event={"ID":"580506e5-2d02-4561-96eb-19619e13ee1b","Type":"ContainerStarted","Data":"db0f2ec4492c59e64032d5f67c69bf5db057ed9cab859cb516856cbbd7dc6d3c"} Jul 16 22:47:28.803438 ip-10-0-136-31 kubenswrapper[2563]: E0716 22:47:28.803414 2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7194-140373846210864\" is forbidden: User \"system:node:ip-10-0-136-31.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-31.ec2.internal' and this object" podUID="ad78efb4-7215-4498-bcf3-dd978aaf4605" pod="opendatahub/test-curl-7194-140373846210864" Jul 16 22:47:29.242281 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:29.242252 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad78efb4-7215-4498-bcf3-dd978aaf4605" path="/var/lib/kubelet/pods/ad78efb4-7215-4498-bcf3-dd978aaf4605/volumes" Jul 16 22:47:29.800493 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:29.800463 2563 generic.go:356] "Generic (PLEG): container finished" podID="580506e5-2d02-4561-96eb-19619e13ee1b" containerID="01f0f640ad2dc57f5aaa3ad5032e84364649cc017aa8ec8b3ed05074e4a0b7e3" exitCode=0 Jul 16 22:47:29.800658 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:29.800541 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846210992" event={"ID":"580506e5-2d02-4561-96eb-19619e13ee1b","Type":"ContainerDied","Data":"01f0f640ad2dc57f5aaa3ad5032e84364649cc017aa8ec8b3ed05074e4a0b7e3"} Jul 16 22:47:30.946184 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:30.946163 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846210992" Jul 16 22:47:30.968518 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:30.968482 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7194-140373846210992_580506e5-2d02-4561-96eb-19619e13ee1b/test-curl-7194-140373846210992/0.log" Jul 16 22:47:30.994679 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:30.994654 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7194-140373846210992"] Jul 16 22:47:30.998716 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:30.998694 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7194-140373846210992"] Jul 16 22:47:31.024799 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.024775 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf56b\" (UniqueName: \"kubernetes.io/projected/580506e5-2d02-4561-96eb-19619e13ee1b-kube-api-access-zf56b\") pod \"580506e5-2d02-4561-96eb-19619e13ee1b\" (UID: \"580506e5-2d02-4561-96eb-19619e13ee1b\") " Jul 16 22:47:31.026743 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.026725 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/580506e5-2d02-4561-96eb-19619e13ee1b-kube-api-access-zf56b" (OuterVolumeSpecName: "kube-api-access-zf56b") pod "580506e5-2d02-4561-96eb-19619e13ee1b" (UID: "580506e5-2d02-4561-96eb-19619e13ee1b"). InnerVolumeSpecName "kube-api-access-zf56b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:47:31.125775 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.125753 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf56b\" (UniqueName: \"kubernetes.io/projected/580506e5-2d02-4561-96eb-19619e13ee1b-kube-api-access-zf56b\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:47:31.169361 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.169340 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:31.169754 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.169740 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="580506e5-2d02-4561-96eb-19619e13ee1b" containerName="test-curl-7194-140373846210992" Jul 16 22:47:31.169797 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.169755 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="580506e5-2d02-4561-96eb-19619e13ee1b" containerName="test-curl-7194-140373846210992" Jul 16 22:47:31.169854 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.169845 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="580506e5-2d02-4561-96eb-19619e13ee1b" containerName="test-curl-7194-140373846210992" Jul 16 22:47:31.172919 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.172905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:31.182697 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.182668 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:31.226969 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.226939 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5l6x\" (UniqueName: \"kubernetes.io/projected/562ad821-f6fc-4393-b406-9fd4e4e6de34-kube-api-access-h5l6x\") pod \"test-curl-7194-140373846082160\" (UID: \"562ad821-f6fc-4393-b406-9fd4e4e6de34\") " pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:31.242012 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.241988 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="580506e5-2d02-4561-96eb-19619e13ee1b" path="/var/lib/kubelet/pods/580506e5-2d02-4561-96eb-19619e13ee1b/volumes" Jul 16 22:47:31.328244 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.328211 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5l6x\" (UniqueName: \"kubernetes.io/projected/562ad821-f6fc-4393-b406-9fd4e4e6de34-kube-api-access-h5l6x\") pod \"test-curl-7194-140373846082160\" (UID: \"562ad821-f6fc-4393-b406-9fd4e4e6de34\") " pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:31.337291 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.337265 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5l6x\" (UniqueName: \"kubernetes.io/projected/562ad821-f6fc-4393-b406-9fd4e4e6de34-kube-api-access-h5l6x\") pod \"test-curl-7194-140373846082160\" (UID: \"562ad821-f6fc-4393-b406-9fd4e4e6de34\") " pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:31.482643 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.482566 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:31.622394 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.622365 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:31.622612 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:47:31.622584 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod562ad821_f6fc_4393_b406_9fd4e4e6de34.slice/crio-2440c7c4be205db54b55d1f9d3e532581682caabe4b5473bf0f4f79ca73e625d WatchSource:0}: Error finding container 2440c7c4be205db54b55d1f9d3e532581682caabe4b5473bf0f4f79ca73e625d: Status 404 returned error can't find the container with id 2440c7c4be205db54b55d1f9d3e532581682caabe4b5473bf0f4f79ca73e625d Jul 16 22:47:31.812869 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.812787 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846210992" Jul 16 22:47:31.812869 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.812834 2563 scope.go:117] "RemoveContainer" containerID="01f0f640ad2dc57f5aaa3ad5032e84364649cc017aa8ec8b3ed05074e4a0b7e3" Jul 16 22:47:31.814113 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:31.814092 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846082160" event={"ID":"562ad821-f6fc-4393-b406-9fd4e4e6de34","Type":"ContainerStarted","Data":"2440c7c4be205db54b55d1f9d3e532581682caabe4b5473bf0f4f79ca73e625d"} Jul 16 22:47:32.822242 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:32.822211 2563 generic.go:356] "Generic (PLEG): container finished" podID="562ad821-f6fc-4393-b406-9fd4e4e6de34" containerID="f03e32b9c5ca83d201a45d8b0bab0cb34e2390e20abe2d8e95afe380454df846" exitCode=0 Jul 16 22:47:32.822628 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:32.822262 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846082160" event={"ID":"562ad821-f6fc-4393-b406-9fd4e4e6de34","Type":"ContainerDied","Data":"f03e32b9c5ca83d201a45d8b0bab0cb34e2390e20abe2d8e95afe380454df846"} Jul 16 22:47:33.957938 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:33.957906 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:33.978459 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:33.978424 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7194-140373846082160_562ad821-f6fc-4393-b406-9fd4e4e6de34/test-curl-7194-140373846082160/0.log" Jul 16 22:47:34.002905 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.002871 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:34.005412 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.005387 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:34.054894 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.054866 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5l6x\" (UniqueName: \"kubernetes.io/projected/562ad821-f6fc-4393-b406-9fd4e4e6de34-kube-api-access-h5l6x\") pod \"562ad821-f6fc-4393-b406-9fd4e4e6de34\" (UID: \"562ad821-f6fc-4393-b406-9fd4e4e6de34\") " Jul 16 22:47:34.056912 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.056882 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/562ad821-f6fc-4393-b406-9fd4e4e6de34-kube-api-access-h5l6x" (OuterVolumeSpecName: "kube-api-access-h5l6x") pod "562ad821-f6fc-4393-b406-9fd4e4e6de34" (UID: "562ad821-f6fc-4393-b406-9fd4e4e6de34"). InnerVolumeSpecName "kube-api-access-h5l6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:47:34.130190 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.130165 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:34.130610 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.130596 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="562ad821-f6fc-4393-b406-9fd4e4e6de34" containerName="test-curl-7194-140373846082160" Jul 16 22:47:34.130667 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.130610 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="562ad821-f6fc-4393-b406-9fd4e4e6de34" containerName="test-curl-7194-140373846082160" Jul 16 22:47:34.130706 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.130673 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="562ad821-f6fc-4393-b406-9fd4e4e6de34" containerName="test-curl-7194-140373846082160" Jul 16 22:47:34.133718 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.133701 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:34.140800 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.140770 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:34.155847 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.155821 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5l6x\" (UniqueName: \"kubernetes.io/projected/562ad821-f6fc-4393-b406-9fd4e4e6de34-kube-api-access-h5l6x\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:47:34.256530 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.256505 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djzbx\" (UniqueName: \"kubernetes.io/projected/5e40d189-8fcb-4f6f-b86e-c12cffdcac94-kube-api-access-djzbx\") pod \"test-curl-7194-140373846082160\" (UID: \"5e40d189-8fcb-4f6f-b86e-c12cffdcac94\") " pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:34.356975 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.356946 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djzbx\" (UniqueName: \"kubernetes.io/projected/5e40d189-8fcb-4f6f-b86e-c12cffdcac94-kube-api-access-djzbx\") pod \"test-curl-7194-140373846082160\" (UID: \"5e40d189-8fcb-4f6f-b86e-c12cffdcac94\") " pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:34.366166 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.366123 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djzbx\" (UniqueName: \"kubernetes.io/projected/5e40d189-8fcb-4f6f-b86e-c12cffdcac94-kube-api-access-djzbx\") pod \"test-curl-7194-140373846082160\" (UID: \"5e40d189-8fcb-4f6f-b86e-c12cffdcac94\") " pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:34.444206 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.444130 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:34.583705 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.583681 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:34.586447 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:47:34.586420 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e40d189_8fcb_4f6f_b86e_c12cffdcac94.slice/crio-64e75b8aa5c4d225d86856637c2819877552351ca4c5fae0d48f0a0678307a66 WatchSource:0}: Error finding container 64e75b8aa5c4d225d86856637c2819877552351ca4c5fae0d48f0a0678307a66: Status 404 returned error can't find the container with id 64e75b8aa5c4d225d86856637c2819877552351ca4c5fae0d48f0a0678307a66 Jul 16 22:47:34.835024 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.834944 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846082160" event={"ID":"5e40d189-8fcb-4f6f-b86e-c12cffdcac94","Type":"ContainerStarted","Data":"64e75b8aa5c4d225d86856637c2819877552351ca4c5fae0d48f0a0678307a66"} Jul 16 22:47:34.836093 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.836071 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2440c7c4be205db54b55d1f9d3e532581682caabe4b5473bf0f4f79ca73e625d" Jul 16 22:47:34.836250 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.836100 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:34.848468 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:34.848436 2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7194-140373846082160" oldPodUID="562ad821-f6fc-4393-b406-9fd4e4e6de34" podUID="5e40d189-8fcb-4f6f-b86e-c12cffdcac94" Jul 16 22:47:35.243364 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:35.243333 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="562ad821-f6fc-4393-b406-9fd4e4e6de34" path="/var/lib/kubelet/pods/562ad821-f6fc-4393-b406-9fd4e4e6de34/volumes" Jul 16 22:47:35.842963 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:35.842927 2563 generic.go:356] "Generic (PLEG): container finished" podID="5e40d189-8fcb-4f6f-b86e-c12cffdcac94" containerID="ce758236f799f538740382bcd9f35be4cced9f5d226bac58020515f8f331c6e4" exitCode=0 Jul 16 22:47:35.843116 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:35.843020 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846082160" event={"ID":"5e40d189-8fcb-4f6f-b86e-c12cffdcac94","Type":"ContainerDied","Data":"ce758236f799f538740382bcd9f35be4cced9f5d226bac58020515f8f331c6e4"} Jul 16 22:47:36.982753 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:36.982726 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:37.005501 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.005480 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7194-140373846082160_5e40d189-8fcb-4f6f-b86e-c12cffdcac94/test-curl-7194-140373846082160/0.log" Jul 16 22:47:37.034233 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.034196 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:37.040095 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.040074 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7194-140373846082160"] Jul 16 22:47:37.080302 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.080279 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djzbx\" (UniqueName: \"kubernetes.io/projected/5e40d189-8fcb-4f6f-b86e-c12cffdcac94-kube-api-access-djzbx\") pod \"5e40d189-8fcb-4f6f-b86e-c12cffdcac94\" (UID: \"5e40d189-8fcb-4f6f-b86e-c12cffdcac94\") " Jul 16 22:47:37.082312 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.082291 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e40d189-8fcb-4f6f-b86e-c12cffdcac94-kube-api-access-djzbx" (OuterVolumeSpecName: "kube-api-access-djzbx") pod "5e40d189-8fcb-4f6f-b86e-c12cffdcac94" (UID: "5e40d189-8fcb-4f6f-b86e-c12cffdcac94"). InnerVolumeSpecName "kube-api-access-djzbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:47:37.163584 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.163556 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:37.164020 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.164006 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e40d189-8fcb-4f6f-b86e-c12cffdcac94" containerName="test-curl-7194-140373846082160" Jul 16 22:47:37.164070 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.164023 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e40d189-8fcb-4f6f-b86e-c12cffdcac94" containerName="test-curl-7194-140373846082160" Jul 16 22:47:37.164110 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.164091 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e40d189-8fcb-4f6f-b86e-c12cffdcac94" containerName="test-curl-7194-140373846082160" Jul 16 22:47:37.167118 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.167103 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:37.174463 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.174431 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:37.181268 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.181245 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djzbx\" (UniqueName: \"kubernetes.io/projected/5e40d189-8fcb-4f6f-b86e-c12cffdcac94-kube-api-access-djzbx\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:47:37.240888 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.240860 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e40d189-8fcb-4f6f-b86e-c12cffdcac94" path="/var/lib/kubelet/pods/5e40d189-8fcb-4f6f-b86e-c12cffdcac94/volumes" Jul 16 22:47:37.282936 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.282879 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb9fv\" (UniqueName: \"kubernetes.io/projected/f049fa4c-763f-4d0d-8242-815e3d1fbf23-kube-api-access-gb9fv\") pod \"test-curl-7194-140373846083168\" (UID: \"f049fa4c-763f-4d0d-8242-815e3d1fbf23\") " pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:37.385257 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.385229 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb9fv\" (UniqueName: \"kubernetes.io/projected/f049fa4c-763f-4d0d-8242-815e3d1fbf23-kube-api-access-gb9fv\") pod \"test-curl-7194-140373846083168\" (UID: \"f049fa4c-763f-4d0d-8242-815e3d1fbf23\") " pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:37.394025 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.393998 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb9fv\" (UniqueName: \"kubernetes.io/projected/f049fa4c-763f-4d0d-8242-815e3d1fbf23-kube-api-access-gb9fv\") pod \"test-curl-7194-140373846083168\" (UID: \"f049fa4c-763f-4d0d-8242-815e3d1fbf23\") " pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:37.478231 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.478164 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:37.615484 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.615455 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:37.616895 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:47:37.616867 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf049fa4c_763f_4d0d_8242_815e3d1fbf23.slice/crio-bb03929c51665a836beccac7819cd1991a397bf7f4fe3ef5e9d9e6c780b3ade1 WatchSource:0}: Error finding container bb03929c51665a836beccac7819cd1991a397bf7f4fe3ef5e9d9e6c780b3ade1: Status 404 returned error can't find the container with id bb03929c51665a836beccac7819cd1991a397bf7f4fe3ef5e9d9e6c780b3ade1 Jul 16 22:47:37.856106 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.856070 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846083168" event={"ID":"f049fa4c-763f-4d0d-8242-815e3d1fbf23","Type":"ContainerStarted","Data":"bb03929c51665a836beccac7819cd1991a397bf7f4fe3ef5e9d9e6c780b3ade1"} Jul 16 22:47:37.857363 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.857344 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846082160" Jul 16 22:47:37.857471 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:37.857393 2563 scope.go:117] "RemoveContainer" containerID="ce758236f799f538740382bcd9f35be4cced9f5d226bac58020515f8f331c6e4" Jul 16 22:47:38.865796 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:38.865758 2563 generic.go:356] "Generic (PLEG): container finished" podID="f049fa4c-763f-4d0d-8242-815e3d1fbf23" containerID="9d22ef1b227e3b5e0663000a48d3cd5868ba9629f4898af350b05041226a8951" exitCode=0 Jul 16 22:47:38.866197 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:38.865816 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846083168" event={"ID":"f049fa4c-763f-4d0d-8242-815e3d1fbf23","Type":"ContainerDied","Data":"9d22ef1b227e3b5e0663000a48d3cd5868ba9629f4898af350b05041226a8951"} Jul 16 22:47:40.019037 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.019015 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:40.043961 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.043940 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7194-140373846083168_f049fa4c-763f-4d0d-8242-815e3d1fbf23/test-curl-7194-140373846083168/0.log" Jul 16 22:47:40.080459 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.080426 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:40.083861 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.083840 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:40.112607 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.112587 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb9fv\" (UniqueName: \"kubernetes.io/projected/f049fa4c-763f-4d0d-8242-815e3d1fbf23-kube-api-access-gb9fv\") pod \"f049fa4c-763f-4d0d-8242-815e3d1fbf23\" (UID: \"f049fa4c-763f-4d0d-8242-815e3d1fbf23\") " Jul 16 22:47:40.114561 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.114536 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f049fa4c-763f-4d0d-8242-815e3d1fbf23-kube-api-access-gb9fv" (OuterVolumeSpecName: "kube-api-access-gb9fv") pod "f049fa4c-763f-4d0d-8242-815e3d1fbf23" (UID: "f049fa4c-763f-4d0d-8242-815e3d1fbf23"). InnerVolumeSpecName "kube-api-access-gb9fv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:47:40.212138 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.212071 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:40.212570 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.212556 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f049fa4c-763f-4d0d-8242-815e3d1fbf23" containerName="test-curl-7194-140373846083168" Jul 16 22:47:40.212609 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.212572 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f049fa4c-763f-4d0d-8242-815e3d1fbf23" containerName="test-curl-7194-140373846083168" Jul 16 22:47:40.212660 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.212650 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f049fa4c-763f-4d0d-8242-815e3d1fbf23" containerName="test-curl-7194-140373846083168" Jul 16 22:47:40.213644 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.213615 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb9fv\" (UniqueName: \"kubernetes.io/projected/f049fa4c-763f-4d0d-8242-815e3d1fbf23-kube-api-access-gb9fv\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:47:40.216381 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.216363 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:40.220753 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.220730 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:40.314879 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.314848 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5q5c\" (UniqueName: \"kubernetes.io/projected/20fe7296-a5c1-49b7-afaa-0e99e6cdaeff-kube-api-access-q5q5c\") pod \"test-curl-7194-140373846083168\" (UID: \"20fe7296-a5c1-49b7-afaa-0e99e6cdaeff\") " pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:40.415720 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.415693 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5q5c\" (UniqueName: \"kubernetes.io/projected/20fe7296-a5c1-49b7-afaa-0e99e6cdaeff-kube-api-access-q5q5c\") pod \"test-curl-7194-140373846083168\" (UID: \"20fe7296-a5c1-49b7-afaa-0e99e6cdaeff\") " pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:40.427022 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.427000 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5q5c\" (UniqueName: \"kubernetes.io/projected/20fe7296-a5c1-49b7-afaa-0e99e6cdaeff-kube-api-access-q5q5c\") pod \"test-curl-7194-140373846083168\" (UID: \"20fe7296-a5c1-49b7-afaa-0e99e6cdaeff\") " pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:40.527403 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.527333 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:40.663280 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.663251 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:40.664773 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:47:40.664752 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20fe7296_a5c1_49b7_afaa_0e99e6cdaeff.slice/crio-d7407ad4e40f6cc08fd88f8a516a10a2d2e16ce5d83162af33cd50661cdd10c5 WatchSource:0}: Error finding container d7407ad4e40f6cc08fd88f8a516a10a2d2e16ce5d83162af33cd50661cdd10c5: Status 404 returned error can't find the container with id d7407ad4e40f6cc08fd88f8a516a10a2d2e16ce5d83162af33cd50661cdd10c5 Jul 16 22:47:40.879222 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.879200 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:40.879362 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.879242 2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb03929c51665a836beccac7819cd1991a397bf7f4fe3ef5e9d9e6c780b3ade1" Jul 16 22:47:40.880417 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.880391 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846083168" event={"ID":"20fe7296-a5c1-49b7-afaa-0e99e6cdaeff","Type":"ContainerStarted","Data":"d7407ad4e40f6cc08fd88f8a516a10a2d2e16ce5d83162af33cd50661cdd10c5"} Jul 16 22:47:40.909356 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:40.909331 2563 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7194-140373846083168" oldPodUID="f049fa4c-763f-4d0d-8242-815e3d1fbf23" podUID="20fe7296-a5c1-49b7-afaa-0e99e6cdaeff" Jul 16 22:47:41.241851 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:41.241772 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f049fa4c-763f-4d0d-8242-815e3d1fbf23" path="/var/lib/kubelet/pods/f049fa4c-763f-4d0d-8242-815e3d1fbf23/volumes" Jul 16 22:47:41.887311 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:41.887272 2563 generic.go:356] "Generic (PLEG): container finished" podID="20fe7296-a5c1-49b7-afaa-0e99e6cdaeff" containerID="0df8abce2086d31bc8f96c938752ebafb8f64e02ec0ecb2d1f4c9a455e588c7d" exitCode=0 Jul 16 22:47:41.887311 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:41.887301 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7194-140373846083168" event={"ID":"20fe7296-a5c1-49b7-afaa-0e99e6cdaeff","Type":"ContainerDied","Data":"0df8abce2086d31bc8f96c938752ebafb8f64e02ec0ecb2d1f4c9a455e588c7d"} Jul 16 22:47:43.048512 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.048491 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:47:43.070653 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.070628 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7194-140373846083168_20fe7296-a5c1-49b7-afaa-0e99e6cdaeff/test-curl-7194-140373846083168/0.log" Jul 16 22:47:43.096217 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.096188 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:43.100216 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.100195 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7194-140373846083168"] Jul 16 22:47:43.142053 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.142030 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5q5c\" (UniqueName: \"kubernetes.io/projected/20fe7296-a5c1-49b7-afaa-0e99e6cdaeff-kube-api-access-q5q5c\") pod \"20fe7296-a5c1-49b7-afaa-0e99e6cdaeff\" (UID: \"20fe7296-a5c1-49b7-afaa-0e99e6cdaeff\") " Jul 16 22:47:43.144076 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.144016 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fe7296-a5c1-49b7-afaa-0e99e6cdaeff-kube-api-access-q5q5c" (OuterVolumeSpecName: "kube-api-access-q5q5c") pod "20fe7296-a5c1-49b7-afaa-0e99e6cdaeff" (UID: "20fe7296-a5c1-49b7-afaa-0e99e6cdaeff"). InnerVolumeSpecName "kube-api-access-q5q5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 22:47:43.241431 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.241407 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20fe7296-a5c1-49b7-afaa-0e99e6cdaeff" path="/var/lib/kubelet/pods/20fe7296-a5c1-49b7-afaa-0e99e6cdaeff/volumes" Jul 16 22:47:43.243073 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.243056 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5q5c\" (UniqueName: \"kubernetes.io/projected/20fe7296-a5c1-49b7-afaa-0e99e6cdaeff-kube-api-access-q5q5c\") on node \"ip-10-0-136-31.ec2.internal\" DevicePath \"\"" Jul 16 22:47:43.901127 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.901098 2563 scope.go:117] "RemoveContainer" containerID="0df8abce2086d31bc8f96c938752ebafb8f64e02ec0ecb2d1f4c9a455e588c7d" Jul 16 22:47:43.901127 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:47:43.901121 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7194-140373846083168" Jul 16 22:48:35.947672 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:35.947631 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-578d6d868b-pcf2f_ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4/authorino/0.log" Jul 16 22:48:41.828697 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:41.828663 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xhrkd_6edbb00f-c805-4dff-8e45-7497cd417530/manager/0.log" Jul 16 22:48:42.084768 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:42.084693 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df6666d-t64df_12738948-2307-42cd-a500-893d2835caa9/manager/0.log" Jul 16 22:48:42.096825 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:42.096806 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df6666d-t64df_12738948-2307-42cd-a500-893d2835caa9/copy-manifests/0.log" Jul 16 22:48:42.615469 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:42.615441 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7746cf446f-qkqtv_84036834-6c09-4d1c-80f9-06c96e1beea9/maas-api/0.log" Jul 16 22:48:44.279096 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:44.279071 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-qmg5t_0c06e1f5-f6e5-47a0-8c1a-fba817fd778e/postgres/0.log" Jul 16 22:48:45.055533 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.055502 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c_3d05f907-88be-4d4b-83ab-fa075e136697/pull/0.log" Jul 16 22:48:45.062499 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.062479 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c_3d05f907-88be-4d4b-83ab-fa075e136697/extract/0.log" Jul 16 22:48:45.069506 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.069483 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c_3d05f907-88be-4d4b-83ab-fa075e136697/util/0.log" Jul 16 22:48:45.185528 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.185501 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl_b6fc0945-783d-4dd7-b85e-57ed897d55c5/util/0.log" Jul 16 22:48:45.192399 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.192378 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl_b6fc0945-783d-4dd7-b85e-57ed897d55c5/pull/0.log" Jul 16 22:48:45.199018 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.199001 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl_b6fc0945-783d-4dd7-b85e-57ed897d55c5/extract/0.log" Jul 16 22:48:45.310214 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.310110 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9_1b892c73-a13d-4248-9ce0-64e8024e4674/extract/0.log" Jul 16 22:48:45.316596 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.316570 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9_1b892c73-a13d-4248-9ce0-64e8024e4674/util/0.log" Jul 16 22:48:45.323305 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.323282 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9_1b892c73-a13d-4248-9ce0-64e8024e4674/pull/0.log" Jul 16 22:48:45.434333 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.434303 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74_46e10611-d199-40ff-8bdf-c2521f06cd97/util/0.log" Jul 16 22:48:45.440605 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.440587 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74_46e10611-d199-40ff-8bdf-c2521f06cd97/pull/0.log" Jul 16 22:48:45.447824 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.447805 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74_46e10611-d199-40ff-8bdf-c2521f06cd97/extract/0.log" Jul 16 22:48:45.565534 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.565451 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-578d6d868b-pcf2f_ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4/authorino/0.log" Jul 16 22:48:45.688787 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:45.688756 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-f2qmt_284b00cf-79a6-4106-9b58-4672485328ae/manager/0.log" Jul 16 22:48:46.031355 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:46.031328 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8jnqf_4b4b6cd2-b946-4036-9550-0500679dd755/registry-server/0.log" Jul 16 22:48:47.224911 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:47.224881 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tvknc_b3ba87a0-de8b-4043-999d-3204f369eaca/istio-proxy/1.log" Jul 16 22:48:47.461914 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:47.461877 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-pwq2p_a333af72-eff1-4360-a8dd-9bfc366bb9ab/payload-processing/0.log" Jul 16 22:48:47.578651 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:47.578576 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-544c55cfbb-t62g9_838fa3a2-b821-45b7-9678-a60fa9e797f2/router/0.log" Jul 16 22:48:47.922493 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:47.922464 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-h2hjl_f32db882-0d9a-43ec-b7a2-8e6ba92bf849/main/0.log" Jul 16 22:48:48.154270 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:48.154238 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-whp4q_b3876ed7-b133-4bdb-a5e7-00b13330fc74/main/0.log" Jul 16 22:48:48.501743 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:48.501714 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-59ktc_acabb8ac-2574-4f31-bcb3-1d2c99423669/main/0.log" Jul 16 22:48:56.989719 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:56.989689 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5p9fm_81422712-0ebb-4177-a94f-11b2b36bc6ab/global-pull-secret-syncer/0.log" Jul 16 22:48:57.180667 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:57.180638 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6s8l9_51fea26c-fe2e-4d70-94c2-03c76aee179c/konnectivity-agent/0.log" Jul 16 22:48:57.252201 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:48:57.252115 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-31.ec2.internal_c1faed2aff0ab49640344e2b4b87931f/haproxy/0.log" Jul 16 22:49:02.581116 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.581086 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c_3d05f907-88be-4d4b-83ab-fa075e136697/extract/0.log" Jul 16 22:49:02.606559 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.606532 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c_3d05f907-88be-4d4b-83ab-fa075e136697/util/0.log" Jul 16 22:49:02.635290 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.635265 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sql4c_3d05f907-88be-4d4b-83ab-fa075e136697/pull/0.log" Jul 16 22:49:02.665776 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.665756 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl_b6fc0945-783d-4dd7-b85e-57ed897d55c5/extract/0.log" Jul 16 22:49:02.688442 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.688419 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl_b6fc0945-783d-4dd7-b85e-57ed897d55c5/util/0.log" Jul 16 22:49:02.715358 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.715335 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bdnkl_b6fc0945-783d-4dd7-b85e-57ed897d55c5/pull/0.log" Jul 16 22:49:02.750165 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.750125 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9_1b892c73-a13d-4248-9ce0-64e8024e4674/extract/0.log" Jul 16 22:49:02.775550 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.775524 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9_1b892c73-a13d-4248-9ce0-64e8024e4674/util/0.log" Jul 16 22:49:02.805726 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.805704 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b8td9_1b892c73-a13d-4248-9ce0-64e8024e4674/pull/0.log" Jul 16 22:49:02.838667 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.838593 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74_46e10611-d199-40ff-8bdf-c2521f06cd97/extract/0.log" Jul 16 22:49:02.865751 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.865728 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74_46e10611-d199-40ff-8bdf-c2521f06cd97/util/0.log" Jul 16 22:49:02.895035 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:02.895016 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q7q74_46e10611-d199-40ff-8bdf-c2521f06cd97/pull/0.log" Jul 16 22:49:03.192849 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:03.192811 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-578d6d868b-pcf2f_ca7bb7cc-2948-49f2-9c5f-d2f1998c39f4/authorino/0.log" Jul 16 22:49:03.230125 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:03.230084 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-f2qmt_284b00cf-79a6-4106-9b58-4672485328ae/manager/0.log" Jul 16 22:49:03.348428 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:03.348400 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-8jnqf_4b4b6cd2-b946-4036-9550-0500679dd755/registry-server/0.log" Jul 16 22:49:05.349950 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.349919 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a566030-98b2-449f-aaed-1e6caaf3556a/alertmanager/0.log" Jul 16 22:49:05.373692 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.373670 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a566030-98b2-449f-aaed-1e6caaf3556a/config-reloader/0.log" Jul 16 22:49:05.404058 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.404037 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a566030-98b2-449f-aaed-1e6caaf3556a/kube-rbac-proxy-web/0.log" Jul 16 22:49:05.428207 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.428178 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a566030-98b2-449f-aaed-1e6caaf3556a/kube-rbac-proxy/0.log" Jul 16 22:49:05.453334 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.453308 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a566030-98b2-449f-aaed-1e6caaf3556a/kube-rbac-proxy-metric/0.log" Jul 16 22:49:05.476183 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.476161 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a566030-98b2-449f-aaed-1e6caaf3556a/prom-label-proxy/0.log" Jul 16 22:49:05.499395 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.499377 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2a566030-98b2-449f-aaed-1e6caaf3556a/init-config-reloader/0.log" Jul 16 22:49:05.541391 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.541354 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-64kl9_25674ffd-9e77-4e5a-aa0f-e2d8ab09c4aa/cluster-monitoring-operator/0.log" Jul 16 22:49:05.643400 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.643339 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-c8c4fdcb9-fhg9d_7f77f1c8-c8d6-4253-8d41-3fd27ca756fa/metrics-server/0.log" Jul 16 22:49:05.680920 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.680896 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-l8pqj_62eab82f-66fd-44ce-8dc0-4b61937eaeee/monitoring-plugin/0.log" Jul 16 22:49:05.789563 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.789536 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv9bc_a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a/node-exporter/0.log" Jul 16 22:49:05.811313 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.811292 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv9bc_a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a/kube-rbac-proxy/0.log" Jul 16 22:49:05.835486 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.835468 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv9bc_a6a3b611-c94b-4f8a-9ddb-a15ac7feb80a/init-textfile/0.log" Jul 16 22:49:05.936002 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.935936 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8t2qp_8e647a04-81bc-44fc-b2f8-084efe58cee9/kube-rbac-proxy-main/0.log" Jul 16 22:49:05.972025 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.972003 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8t2qp_8e647a04-81bc-44fc-b2f8-084efe58cee9/kube-rbac-proxy-self/0.log" Jul 16 22:49:05.999558 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:05.999532 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8t2qp_8e647a04-81bc-44fc-b2f8-084efe58cee9/openshift-state-metrics/0.log" Jul 16 22:49:06.285449 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.285376 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-96r9f_b26f4b90-79eb-4401-884c-570fc4e149c7/prometheus-operator-admission-webhook/0.log" Jul 16 22:49:06.314404 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.314384 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-554b5d6ff9-r7ncn_392f5bf8-1c67-41fb-aed0-c8040f9fefe2/telemeter-client/0.log" Jul 16 22:49:06.337545 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.337524 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-554b5d6ff9-r7ncn_392f5bf8-1c67-41fb-aed0-c8040f9fefe2/reload/0.log" Jul 16 22:49:06.361254 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.361232 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-554b5d6ff9-r7ncn_392f5bf8-1c67-41fb-aed0-c8040f9fefe2/kube-rbac-proxy/0.log" Jul 16 22:49:06.400347 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.400330 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcbc74488-9r7tl_69971137-9cb9-42eb-8b18-5f4e828ff8c1/thanos-query/0.log" Jul 16 22:49:06.429394 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.429377 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcbc74488-9r7tl_69971137-9cb9-42eb-8b18-5f4e828ff8c1/kube-rbac-proxy-web/0.log" Jul 16 22:49:06.454246 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.454226 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcbc74488-9r7tl_69971137-9cb9-42eb-8b18-5f4e828ff8c1/kube-rbac-proxy/0.log" Jul 16 22:49:06.476129 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.476108 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcbc74488-9r7tl_69971137-9cb9-42eb-8b18-5f4e828ff8c1/prom-label-proxy/0.log" Jul 16 22:49:06.501622 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.501605 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcbc74488-9r7tl_69971137-9cb9-42eb-8b18-5f4e828ff8c1/kube-rbac-proxy-rules/0.log" Jul 16 22:49:06.524582 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.524566 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7dcbc74488-9r7tl_69971137-9cb9-42eb-8b18-5f4e828ff8c1/kube-rbac-proxy-metrics/0.log" Jul 16 22:49:06.870788 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.870759 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn"] Jul 16 22:49:06.871224 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.871209 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fe7296-a5c1-49b7-afaa-0e99e6cdaeff" containerName="test-curl-7194-140373846083168" Jul 16 22:49:06.871224 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.871225 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fe7296-a5c1-49b7-afaa-0e99e6cdaeff" containerName="test-curl-7194-140373846083168" Jul 16 22:49:06.871318 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.871299 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fe7296-a5c1-49b7-afaa-0e99e6cdaeff" containerName="test-curl-7194-140373846083168" Jul 16 22:49:06.874487 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.874467 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:06.879928 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.879889 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nx7gf\"/\"default-dockercfg-pvwf8\"" Jul 16 22:49:06.880235 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.880214 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nx7gf\"/\"openshift-service-ca.crt\"" Jul 16 22:49:06.881538 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.881519 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nx7gf\"/\"kube-root-ca.crt\"" Jul 16 22:49:06.886868 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:06.886849 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn"] Jul 16 22:49:07.011493 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.011450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn4qn\" (UniqueName: \"kubernetes.io/projected/43366859-4d03-410c-bd6e-d4af55745b97-kube-api-access-kn4qn\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.011658 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.011566 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-proc\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.011658 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.011592 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-podres\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.011738 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.011667 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-lib-modules\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.011773 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.011737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-sys\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112157 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112116 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-proc\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112293 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112166 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-podres\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112293 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112242 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-proc\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112293 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112285 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-lib-modules\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112393 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112362 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-sys\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112428 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112403 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-sys\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112428 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112287 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-podres\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112494 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112366 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/43366859-4d03-410c-bd6e-d4af55745b97-lib-modules\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.112494 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.112402 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn4qn\" (UniqueName: \"kubernetes.io/projected/43366859-4d03-410c-bd6e-d4af55745b97-kube-api-access-kn4qn\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.121634 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.121574 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn4qn\" (UniqueName: \"kubernetes.io/projected/43366859-4d03-410c-bd6e-d4af55745b97-kube-api-access-kn4qn\") pod \"perf-node-gather-daemonset-7gvvn\" (UID: \"43366859-4d03-410c-bd6e-d4af55745b97\") " pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.184828 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.184803 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:07.346100 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.346070 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn"] Jul 16 22:49:07.347598 ip-10-0-136-31 kubenswrapper[2563]: W0716 22:49:07.347574 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod43366859_4d03_410c_bd6e_d4af55745b97.slice/crio-e773e67939cc03c53a83b3086465e25d798e410b6969abfc31c301c7771e79ad WatchSource:0}: Error finding container e773e67939cc03c53a83b3086465e25d798e410b6969abfc31c301c7771e79ad: Status 404 returned error can't find the container with id e773e67939cc03c53a83b3086465e25d798e410b6969abfc31c301c7771e79ad Jul 16 22:49:07.395200 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:07.395173 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" event={"ID":"43366859-4d03-410c-bd6e-d4af55745b97","Type":"ContainerStarted","Data":"e773e67939cc03c53a83b3086465e25d798e410b6969abfc31c301c7771e79ad"} Jul 16 22:49:08.195314 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:08.195286 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/1.log" Jul 16 22:49:08.200697 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:08.200675 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8vbpp_d91210e8-ef77-43ac-8287-c51e70dddbd9/console-operator/2.log" Jul 16 22:49:08.404098 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:08.404046 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" event={"ID":"43366859-4d03-410c-bd6e-d4af55745b97","Type":"ContainerStarted","Data":"343c705b7bf15bc4c8fffbe8ca10fd49a210ea28010e15ba1f895a1ad2500a59"} Jul 16 22:49:08.404499 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:08.404305 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:08.425934 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:08.425879 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" podStartSLOduration=2.425863907 podStartE2EDuration="2.425863907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 22:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 22:49:08.424440234 +0000 UTC m=+3743.757519287" watchObservedRunningTime="2026-07-16 22:49:08.425863907 +0000 UTC m=+3743.758942937" Jul 16 22:49:08.672587 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:08.672553 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-66d4d88769-hlbxd_ee3b4e03-9dd6-49ca-aebb-8df4cb9f4272/console/0.log" Jul 16 22:49:08.703843 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:08.703813 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-j8k94_56a8954a-62c7-45c7-92dd-fe4421a8fe9f/download-server/0.log" Jul 16 22:49:10.468922 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:10.468853 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vms7h_963b9f29-793e-4f12-a7bc-426a7b06dd53/dns/0.log" Jul 16 22:49:10.488974 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:10.488947 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vms7h_963b9f29-793e-4f12-a7bc-426a7b06dd53/kube-rbac-proxy/0.log" Jul 16 22:49:10.536695 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:10.536676 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fpm78_c3dbc476-9c43-48c3-a2bf-78a67110c028/dns-node-resolver/0.log" Jul 16 22:49:11.206672 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:11.206644 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pzmdz_a160cbfd-0e76-4746-9d1a-e442b9dbdff1/node-ca/0.log" Jul 16 22:49:12.711203 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:12.711177 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tvknc_b3ba87a0-de8b-4043-999d-3204f369eaca/istio-proxy/0.log" Jul 16 22:49:12.739692 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:12.739666 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tvknc_b3ba87a0-de8b-4043-999d-3204f369eaca/istio-proxy/1.log" Jul 16 22:49:12.829496 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:12.829470 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-pwq2p_a333af72-eff1-4360-a8dd-9bfc366bb9ab/payload-processing/0.log" Jul 16 22:49:12.869266 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:12.869246 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-544c55cfbb-t62g9_838fa3a2-b821-45b7-9678-a60fa9e797f2/router/0.log" Jul 16 22:49:13.628532 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:13.628500 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f22kb_d366d608-8179-4e70-8529-6dce72b0b665/serve-healthcheck-canary/0.log" Jul 16 22:49:14.123729 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:14.123696 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-prwdk_81e44120-3c18-4e07-ba65-7ce68c3255e8/insights-operator/0.log" Jul 16 22:49:14.124498 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:14.124480 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-prwdk_81e44120-3c18-4e07-ba65-7ce68c3255e8/insights-operator/1.log" Jul 16 22:49:14.301895 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:14.301868 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5c8c_bad1b8fe-5a74-4b1c-a740-06a9c49128c0/kube-rbac-proxy/0.log" Jul 16 22:49:14.325668 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:14.325641 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5c8c_bad1b8fe-5a74-4b1c-a740-06a9c49128c0/exporter/0.log" Jul 16 22:49:14.348678 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:14.348654 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5c8c_bad1b8fe-5a74-4b1c-a740-06a9c49128c0/extractor/0.log" Jul 16 22:49:14.420581 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:14.420515 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nx7gf/perf-node-gather-daemonset-7gvvn" Jul 16 22:49:17.278690 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:17.278597 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xhrkd_6edbb00f-c805-4dff-8e45-7497cd417530/manager/0.log" Jul 16 22:49:17.380771 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:17.380740 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df6666d-t64df_12738948-2307-42cd-a500-893d2835caa9/manager/0.log" Jul 16 22:49:17.406352 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:17.406327 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df6666d-t64df_12738948-2307-42cd-a500-893d2835caa9/copy-manifests/0.log" Jul 16 22:49:17.435059 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:17.435036 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tvknc_b3ba87a0-de8b-4043-999d-3204f369eaca/istio-proxy/0.log" Jul 16 22:49:17.464825 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:17.464789 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tvknc_b3ba87a0-de8b-4043-999d-3204f369eaca/istio-proxy/1.log" Jul 16 22:49:17.519104 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:17.519071 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-pwq2p_a333af72-eff1-4360-a8dd-9bfc366bb9ab/payload-processing/0.log" Jul 16 22:49:17.537132 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:17.537057 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-544c55cfbb-t62g9_838fa3a2-b821-45b7-9678-a60fa9e797f2/router/0.log" Jul 16 22:49:18.974161 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:18.974098 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6c6c96995f-mhmwt_577542db-ff6d-4a89-a68d-5031a26fbbb4/manager/0.log" Jul 16 22:49:19.045305 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:19.045259 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-nf8ww_ebda4572-5df2-42cf-a5ce-4c4521edaf49/openshift-lws-operator/0.log" Jul 16 22:49:19.113003 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:19.112975 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tvknc_b3ba87a0-de8b-4043-999d-3204f369eaca/istio-proxy/0.log" Jul 16 22:49:19.142843 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:19.142817 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tvknc_b3ba87a0-de8b-4043-999d-3204f369eaca/istio-proxy/1.log" Jul 16 22:49:19.192116 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:19.192090 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-pwq2p_a333af72-eff1-4360-a8dd-9bfc366bb9ab/payload-processing/0.log" Jul 16 22:49:19.206070 ip-10-0-136-31 kubenswrapper[2563]: I0716 22:49:19.206047 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-544c55cfbb-t62g9_838fa3a2-b821-45b7-9678-a60fa9e797f2/router/0.log"