Jul 12 17:51:29.846100 ip-10-0-143-136 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 12 17:51:29.846112 ip-10-0-143-136 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 12 17:51:29.846119 ip-10-0-143-136 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 12 17:51:29.846406 ip-10-0-143-136 systemd[1]: Failed to start Kubernetes Kubelet. Jul 12 17:51:39.893330 ip-10-0-143-136 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 12 17:51:39.893344 ip-10-0-143-136 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cc4d0eb228e24c209a15eefa9dfebfa1 -- Jul 12 17:53:26.251915 ip-10-0-143-136 systemd[1]: Starting Kubernetes Kubelet... Jul 12 17:53:26.603447 ip-10-0-143-136 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 17:53:26.603447 ip-10-0-143-136 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 17:53:26.603447 ip-10-0-143-136 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 17:53:26.603447 ip-10-0-143-136 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 17:53:26.603447 ip-10-0-143-136 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 17:53:26.604392 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.604296 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 17:53:26.608821 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608803 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:26.608821 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608819 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:26.608821 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608824 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:26.608821 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608829 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608833 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608838 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608843 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608848 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608852 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608860 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608865 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608870 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608874 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608880 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608884 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608889 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608893 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608898 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608902 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608907 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608911 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608915 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:26.609081 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608919 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608923 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608927 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608931 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608935 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608940 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608944 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608948 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608952 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608959 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608966 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608970 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608975 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608980 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608984 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608990 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608994 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.608999 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609004 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:26.609902 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609008 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609013 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609018 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609023 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609028 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609033 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609037 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609041 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609046 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609050 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609055 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609059 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609064 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609069 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609073 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609077 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609082 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609088 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609093 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609097 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:26.610837 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609102 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609106 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609111 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609116 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609120 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609125 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609129 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609136 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609144 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609150 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609155 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609160 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609165 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609170 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609175 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609180 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609184 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609189 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609194 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609199 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:26.611731 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609203 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609210 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609215 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609220 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609224 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609228 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609234 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609239 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609243 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609248 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609252 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609257 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609261 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609265 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609270 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609274 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609279 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.609284 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610547 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:26.612660 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610557 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610563 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610568 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610572 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610577 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610601 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610606 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610610 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610615 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610619 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610623 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610629 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610634 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610638 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610643 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610647 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610652 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610656 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610660 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610664 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:26.613501 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610669 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610674 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610678 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610682 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610687 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610691 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610697 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610702 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610709 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610716 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610721 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610726 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610732 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610738 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610743 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610747 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610752 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610757 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610762 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610766 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:26.614327 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610771 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610775 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610780 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610785 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610790 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610796 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610801 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610805 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610810 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610814 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610819 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610823 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610828 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610832 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610837 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610841 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610845 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610849 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610854 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:26.615035 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610860 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610867 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610872 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610876 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610884 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610889 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610894 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610898 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610903 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610907 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610912 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610916 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610921 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610926 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610934 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610940 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610945 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610950 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610955 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:26.615693 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610960 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610965 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610970 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610974 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610979 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610983 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610987 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610992 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.610996 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611001 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611005 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611010 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611014 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611019 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611023 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611028 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611033 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611037 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611042 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611047 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:26.616187 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611758 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611768 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611774 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611779 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611783 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611788 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611792 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611798 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611802 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611806 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611813 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611821 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611826 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611831 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611836 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611841 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611846 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611852 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611858 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:26.616812 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611863 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611868 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611873 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611878 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611883 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611888 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611894 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611898 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611904 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611909 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611914 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611919 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611924 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611929 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611934 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611939 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611944 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611948 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611953 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611958 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:26.617437 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611962 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611967 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611971 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611977 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611982 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611987 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611992 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.611996 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612001 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612006 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612011 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612018 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612023 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612028 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612033 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612038 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612044 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612049 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612053 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612058 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:26.617985 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612064 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612069 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612074 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612079 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612083 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612088 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612093 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612098 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612102 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612106 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612111 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612115 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612119 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612124 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612129 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612133 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612139 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612143 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612148 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612153 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:26.618504 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612157 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612162 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612170 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612177 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612181 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612185 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612190 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612194 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612199 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612203 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612208 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612213 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612217 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612222 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612226 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612231 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612237 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612243 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612247 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:26.619040 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.612251 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612354 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612365 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612376 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612383 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612390 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612395 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612402 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612409 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612415 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612420 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612426 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612432 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612438 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612444 2568 flags.go:64] FLAG: --cgroup-root="" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612453 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612460 2568 flags.go:64] FLAG: --client-ca-file="" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612465 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612470 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612477 2568 flags.go:64] FLAG: --cluster-domain="" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612482 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612487 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612492 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612498 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 17:53:26.619524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612505 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612510 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612516 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612522 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612527 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612532 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612538 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612543 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612548 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612555 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612561 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612565 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612570 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612575 2568 flags.go:64] FLAG: --enable-server="true" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612599 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612608 2568 flags.go:64] FLAG: --event-burst="100" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612614 2568 flags.go:64] FLAG: --event-qps="50" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612619 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612624 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612629 2568 flags.go:64] FLAG: --eviction-hard="" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612636 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612642 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612648 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612656 2568 flags.go:64] FLAG: --eviction-soft="" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612663 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 17:53:26.620229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612668 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612673 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612678 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612683 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612688 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612694 2568 flags.go:64] FLAG: --feature-gates="" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612701 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612707 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612712 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612717 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612722 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612728 2568 flags.go:64] FLAG: --help="false" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612733 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-143-136.ec2.internal" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612738 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612743 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612748 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612755 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612761 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612766 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612771 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612775 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612781 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612786 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612792 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 17:53:26.620962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612797 2568 flags.go:64] FLAG: --kube-reserved="" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612802 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612807 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612812 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612817 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612824 2568 flags.go:64] FLAG: --lock-file="" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612831 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612838 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612843 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612852 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612858 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612862 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612868 2568 flags.go:64] FLAG: --logging-format="text" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612873 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612879 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612884 2568 flags.go:64] FLAG: --manifest-url="" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612890 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612897 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612902 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612909 2568 flags.go:64] FLAG: --max-pods="110" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612914 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612920 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612924 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612929 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612935 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 17:53:26.621989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612940 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612945 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612957 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612962 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612967 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612973 2568 flags.go:64] FLAG: --pod-cidr="" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612977 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612986 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612991 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.612997 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613001 2568 flags.go:64] FLAG: --port="10250" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613007 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613012 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d881d7b5dda13cf7" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613020 2568 flags.go:64] FLAG: --qos-reserved="" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613026 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613032 2568 flags.go:64] FLAG: --register-node="true" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613037 2568 flags.go:64] FLAG: --register-with-taints="" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613049 2568 flags.go:64] FLAG: --registry-burst="10" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613054 2568 flags.go:64] FLAG: --registry-qps="5" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613059 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613064 2568 flags.go:64] FLAG: --reserved-memory="" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613070 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613075 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613081 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613087 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 17:53:26.622748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613092 2568 flags.go:64] FLAG: --runonce="false" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613097 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613102 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613108 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613113 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613118 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613123 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613128 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613134 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613139 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613144 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613149 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613153 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613159 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613164 2568 flags.go:64] FLAG: --system-cgroups="" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613169 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613178 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613183 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613188 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613195 2568 flags.go:64] FLAG: --tls-min-version="" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613202 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613207 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613214 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613221 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613226 2568 flags.go:64] FLAG: --v="2" Jul 12 17:53:26.623395 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613233 2568 flags.go:64] FLAG: --version="false" Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613240 2568 flags.go:64] FLAG: --vmodule="" Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613246 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613252 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613444 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613451 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613456 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613461 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613468 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613473 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613477 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613482 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613487 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613492 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613496 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613500 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613505 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613509 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613514 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613518 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:26.624061 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613522 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613527 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613532 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613536 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613541 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613545 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613550 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613556 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613561 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613568 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613573 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613577 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613598 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613603 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613608 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613613 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613618 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613623 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613627 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613631 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:26.624599 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613636 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613640 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613644 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613649 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613654 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613659 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613663 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613668 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613673 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613677 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613682 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613686 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613690 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613695 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613699 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613704 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613708 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613713 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613717 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613724 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:26.625162 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613728 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613735 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613744 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613749 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613754 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613758 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613763 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613767 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613772 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613777 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613782 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613787 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613791 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613796 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613801 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613805 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613809 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613816 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613822 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:26.625704 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613827 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613831 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613836 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613841 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613845 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613850 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613854 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613859 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613863 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613868 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613873 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613878 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613888 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613893 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613900 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613907 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613914 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613920 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613925 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:26.626190 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613930 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613935 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613940 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613945 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.613950 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.613959 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.622622 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.622637 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.622658 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.622665 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622723 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622729 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622733 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622736 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622739 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:26.626703 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622742 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622745 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622749 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622752 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622755 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622757 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622762 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622766 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622770 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622774 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622778 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622781 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622784 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622787 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622790 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622793 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622796 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622798 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622801 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:26.627104 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622805 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622807 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622810 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622813 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622816 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622819 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622824 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622828 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622831 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622835 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622838 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622841 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622844 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622847 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622850 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622853 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622856 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622859 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622862 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:26.627611 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622865 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622868 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622871 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622873 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622876 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622879 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622882 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622885 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622887 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622890 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622893 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622895 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622898 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622901 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622904 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622906 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622909 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622912 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622915 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622918 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:26.628096 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622921 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622924 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622926 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622929 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622932 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622934 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622938 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622942 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622945 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622948 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622951 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622954 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622956 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622959 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622962 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622964 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622968 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622970 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622973 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:26.628628 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622976 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622979 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622981 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622984 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622987 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622989 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622992 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622995 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.622997 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623000 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623003 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623007 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623010 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623013 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623016 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623019 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:26.629131 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623021 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.623027 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623144 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623149 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623153 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623158 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623162 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623165 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623168 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623171 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623175 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623178 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623181 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623184 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623188 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 17:53:26.629556 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623190 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623193 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623196 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623199 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623202 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623204 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623207 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623210 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623212 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623215 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623218 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623221 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623224 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623227 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623230 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623233 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623236 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623239 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623241 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623244 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 17:53:26.629962 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623247 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623249 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623252 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623257 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623260 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623263 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623266 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623269 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623272 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623275 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623278 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623281 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623284 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623287 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623289 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623292 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623294 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623297 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623300 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 17:53:26.630477 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623303 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623306 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623308 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623311 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623314 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623317 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623320 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623322 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623325 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623328 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623331 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623333 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623336 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623338 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623341 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623344 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623346 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623349 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623352 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623354 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 17:53:26.631122 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623357 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623359 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623362 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623365 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623367 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623370 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623373 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623377 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623380 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623384 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623387 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623390 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623393 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623396 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623399 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623402 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623406 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623409 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623413 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623416 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 17:53:26.631648 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623419 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623422 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623425 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623428 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623431 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623434 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:26.623436 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.623442 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.623566 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.627963 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.630139 2568 server.go:1019] "Starting client certificate rotation" Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.630238 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 17:53:26.632151 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.630271 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 17:53:26.632495 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.632481 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 17:53:26.633807 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.633789 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 17:53:26.643721 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.643700 2568 log.go:25] "Validated CRI v1 runtime API" Jul 12 17:53:26.648895 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.648879 2568 log.go:25] "Validated CRI v1 image API" Jul 12 17:53:26.649762 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.649748 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 17:53:26.652111 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.652086 2568 fs.go:135] Filesystem UUIDs: map[44f86773-b53f-48c5-ba71-13421083d5f7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e7745010-71e8-41aa-8b7c-c619a671610d:/dev/nvme0n1p4] Jul 12 17:53:26.652164 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.652112 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 17:53:26.658455 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.658333 2568 manager.go:216] Machine: {Timestamp:2026-07-12 17:53:26.656261076 +0000 UTC m=+0.300103794 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100016 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e2b10ed787648ccaede9132a53d10 SystemUUID:ec2e2b10-ed78-7648-ccae-de9132a53d10 BootID:cc4d0eb2-28e2-4c20-9a15-eefa9dfebfa1 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9a:d3:08:96:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9a:d3:08:96:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:f9:1a:12:6d:70 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 17:53:26.658455 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.658452 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 17:53:26.658600 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.658542 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 17:53:26.659969 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.659953 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 17:53:26.661744 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.661720 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 17:53:26.661892 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.661746 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-136.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 17:53:26.662492 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.662483 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 17:53:26.662540 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.662495 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 17:53:26.662540 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.662504 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 17:53:26.663177 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.663167 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 17:53:26.664666 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.664655 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 17:53:26.669015 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669003 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 12 17:53:26.669152 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669142 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 17:53:26.669188 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669173 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 12 17:53:26.669188 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669184 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 17:53:26.669255 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669207 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 17:53:26.669255 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669216 2568 kubelet.go:398] "Adding apiserver pod source" Jul 12 17:53:26.669320 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669262 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 17:53:26.669924 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669906 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 17:53:26.669981 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.669939 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 17:53:26.673167 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.673140 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 17:53:26.675068 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675054 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 17:53:26.675124 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675087 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 17:53:26.675266 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675255 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 17:53:26.675297 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675272 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 17:53:26.675297 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675278 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 17:53:26.675297 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675284 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 17:53:26.675386 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675303 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 17:53:26.675386 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675309 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 17:53:26.675820 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675809 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 17:53:26.675854 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675820 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 17:53:26.675854 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675828 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 17:53:26.675854 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675834 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 17:53:26.675854 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675843 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 17:53:26.675854 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.675851 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 17:53:26.676612 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.676603 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 17:53:26.676612 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.676612 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 17:53:26.680116 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.680103 2568 server.go:1268] "Started kubelet" Jul 12 17:53:26.680231 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.680183 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 17:53:26.680288 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.680255 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 17:53:26.680363 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.680326 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 17:53:26.680972 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.680952 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-136.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 17:53:26.681008 ip-10-0-143-136 systemd[1]: Started Kubernetes Kubelet. Jul 12 17:53:26.681461 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.681434 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 17:53:26.681556 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.681432 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-136.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 17:53:26.681874 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.681850 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 17:53:26.682761 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.682745 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 12 17:53:26.685073 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.685053 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 17:53:26.687243 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.687223 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lbhc2" Jul 12 17:53:26.687910 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.687888 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 17:53:26.687994 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.687908 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 17:53:26.688135 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.688082 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:26.688565 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.688541 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 17:53:26.688648 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.688535 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 17:53:26.688648 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.688636 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 17:53:26.688766 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.688742 2568 factory.go:55] Registering systemd factory Jul 12 17:53:26.688766 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.688761 2568 factory.go:223] Registration of the systemd container factory successfully Jul 12 17:53:26.688861 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.688784 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 12 17:53:26.688861 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.688792 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 12 17:53:26.689060 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.689046 2568 factory.go:153] Registering CRI-O factory Jul 12 17:53:26.689115 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.689068 2568 factory.go:223] Registration of the crio container factory successfully Jul 12 17:53:26.689115 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.689120 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 17:53:26.689214 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.689143 2568 factory.go:103] Registering Raw factory Jul 12 17:53:26.689214 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.689160 2568 manager.go:1196] Started watching for new ooms in manager Jul 12 17:53:26.689905 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.689891 2568 manager.go:349] Starting recovery of all containers Jul 12 17:53:26.689998 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.689085 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-136.ec2.internal.18c19c52c2837003 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-136.ec2.internal,UID:ip-10-0-143-136.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-136.ec2.internal,},FirstTimestamp:2026-07-12 17:53:26.680072195 +0000 UTC m=+0.323914912,LastTimestamp:2026-07-12 17:53:26.680072195 +0000 UTC m=+0.323914912,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-136.ec2.internal,}" Jul 12 17:53:26.694185 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.694152 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 12 17:53:26.694457 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.694428 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-136.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 12 17:53:26.695407 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.695380 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lbhc2" Jul 12 17:53:26.706094 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.705936 2568 manager.go:354] Recovery completed Jul 12 17:53:26.710577 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.710558 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:26.712486 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.712469 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:26.712576 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.712497 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:26.712576 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.712508 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:26.713100 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.713079 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 17:53:26.713100 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.713098 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 17:53:26.713211 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.713118 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 12 17:53:26.713211 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.713088 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:26.713986 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.713966 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:26.714081 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.713997 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:26.714081 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.714010 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:26.714081 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.713966 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-136.ec2.internal.18c19c52c471ff80 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-136.ec2.internal,UID:ip-10-0-143-136.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-136.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-136.ec2.internal,},FirstTimestamp:2026-07-12 17:53:26.712483712 +0000 UTC m=+0.356326432,LastTimestamp:2026-07-12 17:53:26.712483712 +0000 UTC m=+0.356326432,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-136.ec2.internal,}" Jul 12 17:53:26.715439 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.715426 2568 policy_none.go:49] "None policy: Start" Jul 12 17:53:26.715497 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.715442 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 17:53:26.715497 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.715454 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 17:53:26.717604 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.717578 2568 policy_none.go:47] "Start" Jul 12 17:53:26.752238 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.752219 2568 manager.go:337] "Starting Device Plugin manager" Jul 12 17:53:26.752307 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.752253 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 17:53:26.752307 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.752266 2568 server.go:87] "Starting device plugin registration server" Jul 12 17:53:26.752623 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.752578 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 17:53:26.752667 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.752625 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 17:53:26.752764 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.752741 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 17:53:26.752886 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.752868 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 17:53:26.753011 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.752887 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 17:53:26.753415 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.753394 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 17:53:26.753493 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.753450 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:26.827120 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.827070 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 17:53:26.829579 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.828543 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 17:53:26.829579 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.828604 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 17:53:26.829579 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.828638 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 17:53:26.829579 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.828675 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 17:53:26.831278 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.831253 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:26.853382 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.853365 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:26.854513 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.854458 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:26.854513 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.854504 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:26.854681 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.854522 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:26.854681 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.854592 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-136.ec2.internal" Jul 12 17:53:26.863315 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.863299 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-136.ec2.internal" Jul 12 17:53:26.863401 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.863324 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-136.ec2.internal\": node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:26.878113 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.878091 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:26.929217 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.929178 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal"] Jul 12 17:53:26.929307 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.929262 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:26.930241 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.930225 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:26.930297 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.930255 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:26.930297 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.930268 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:26.932883 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.932865 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:26.933003 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.932987 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:26.933049 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.933024 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:26.933630 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.933614 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:26.933702 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.933640 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:26.933702 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.933662 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:26.933702 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.933673 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:26.933802 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.933642 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:26.933802 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.933741 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:26.936166 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.936151 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" Jul 12 17:53:26.936264 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.936174 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 17:53:26.936809 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.936792 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientMemory" Jul 12 17:53:26.936888 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.936820 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 17:53:26.936888 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:26.936831 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeHasSufficientPID" Jul 12 17:53:26.957351 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.957330 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-136.ec2.internal\" not found" node="ip-10-0-143-136.ec2.internal" Jul 12 17:53:26.962085 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.962065 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-136.ec2.internal\" not found" node="ip-10-0-143-136.ec2.internal" Jul 12 17:53:26.979407 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:26.979386 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:27.079901 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:27.079862 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:27.090387 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.090358 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc6ece70f95fe4adf85e0625be1d790d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal\" (UID: \"dc6ece70f95fe4adf85e0625be1d790d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.090491 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.090391 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc6ece70f95fe4adf85e0625be1d790d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal\" (UID: \"dc6ece70f95fe4adf85e0625be1d790d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.090491 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.090409 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73fc1951afb982df9c53cf26d8ff4718-config\") pod \"kube-apiserver-proxy-ip-10-0-143-136.ec2.internal\" (UID: \"73fc1951afb982df9c53cf26d8ff4718\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.181132 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:27.181044 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:27.191120 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.191094 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc6ece70f95fe4adf85e0625be1d790d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal\" (UID: \"dc6ece70f95fe4adf85e0625be1d790d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.191210 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.191129 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc6ece70f95fe4adf85e0625be1d790d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal\" (UID: \"dc6ece70f95fe4adf85e0625be1d790d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.191210 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.191154 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73fc1951afb982df9c53cf26d8ff4718-config\") pod \"kube-apiserver-proxy-ip-10-0-143-136.ec2.internal\" (UID: \"73fc1951afb982df9c53cf26d8ff4718\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.191318 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.191194 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dc6ece70f95fe4adf85e0625be1d790d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal\" (UID: \"dc6ece70f95fe4adf85e0625be1d790d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.191318 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.191201 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc6ece70f95fe4adf85e0625be1d790d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal\" (UID: \"dc6ece70f95fe4adf85e0625be1d790d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.191318 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.191282 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/73fc1951afb982df9c53cf26d8ff4718-config\") pod \"kube-apiserver-proxy-ip-10-0-143-136.ec2.internal\" (UID: \"73fc1951afb982df9c53cf26d8ff4718\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.263808 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.263775 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.264954 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.264936 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.281664 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:27.281631 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:27.386453 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:27.386402 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:27.494095 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:27.494007 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:27.520833 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.520805 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:27.594436 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:27.594397 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:27.629709 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.629665 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 17:53:27.630364 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.629846 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 17:53:27.630364 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.629896 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 17:53:27.694640 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:27.694605 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-136.ec2.internal\" not found" Jul 12 17:53:27.703657 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.703617 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 17:48:27 +0000 UTC" deadline="2028-04-28 05:27:40.622466504 +0000 UTC" Jul 12 17:53:27.703657 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.703651 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15731h34m12.918819457s" Jul 12 17:53:27.733252 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.733222 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:27.789022 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.788958 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.797579 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.797558 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 17:53:27.799167 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.799143 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" Jul 12 17:53:27.809599 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.809580 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 17:53:27.952381 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.952353 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:27.952993 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:27.952966 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc6ece70f95fe4adf85e0625be1d790d.slice/crio-c2748d61b58bab8a4396d1a118600e67f43d94ef19cbed746b650d4284022cda WatchSource:0}: Error finding container c2748d61b58bab8a4396d1a118600e67f43d94ef19cbed746b650d4284022cda: Status 404 returned error can't find the container with id c2748d61b58bab8a4396d1a118600e67f43d94ef19cbed746b650d4284022cda Jul 12 17:53:27.962973 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:27.962958 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 17:53:27.968725 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:27.968696 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fc1951afb982df9c53cf26d8ff4718.slice/crio-780092d408826e6bfd4d38c20710c52c3b38925cfa99db3a26360da1df9ccea7 WatchSource:0}: Error finding container 780092d408826e6bfd4d38c20710c52c3b38925cfa99db3a26360da1df9ccea7: Status 404 returned error can't find the container with id 780092d408826e6bfd4d38c20710c52c3b38925cfa99db3a26360da1df9ccea7 Jul 12 17:53:28.671233 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.671153 2568 apiserver.go:52] "Watching apiserver" Jul 12 17:53:28.682448 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.681206 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 17:53:28.683153 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.683122 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-d7cld","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal","openshift-multus/multus-additional-cni-plugins-wxmk8","openshift-multus/multus-ww5j4","openshift-network-diagnostics/network-check-target-cwgzb","openshift-network-operator/iptables-alerter-pwng9","kube-system/konnectivity-agent-dnw88","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k","openshift-cluster-node-tuning-operator/tuned-gt9wl","openshift-image-registry/node-ca-498ks","openshift-multus/network-metrics-daemon-nxhsg","openshift-ovn-kubernetes/ovnkube-node-4j476","kube-system/global-pull-secret-syncer-djndg","kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal"] Jul 12 17:53:28.686019 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.685915 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:28.688601 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.688571 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.689278 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.689241 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:28.689410 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.689388 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rf5jb\"" Jul 12 17:53:28.689681 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.689661 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 17:53:28.689809 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.689791 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 17:53:28.689889 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.689847 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 17:53:28.690722 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.690703 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.693360 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.693341 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.694295 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.694277 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 17:53:28.694295 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.694286 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 17:53:28.694473 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.694454 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 17:53:28.694563 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.694530 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 17:53:28.694863 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.694689 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qw8kl\"" Jul 12 17:53:28.694863 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.694758 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 17:53:28.695624 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.695601 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 17:53:28.695976 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.695943 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:28.696067 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.695985 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9fd42\"" Jul 12 17:53:28.696067 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.696020 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:28.698515 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.698496 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.700311 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700286 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d907cd33-4123-4333-a38a-ac8e60b5ef04-kubelet-config\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.700410 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700335 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-os-release\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.700410 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.700410 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700389 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df4k7\" (UniqueName: \"kubernetes.io/projected/950b9259-0b9e-4847-838d-26aad602908c-kube-api-access-df4k7\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.700575 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700415 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d907cd33-4123-4333-a38a-ac8e60b5ef04-dbus\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.700575 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700446 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-cni-binary-copy\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.700575 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700472 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.700575 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700505 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3322fe7-5ad0-4f6d-9df7-300b067e2082-agent-certs\") pod \"konnectivity-agent-dnw88\" (UID: \"b3322fe7-5ad0-4f6d-9df7-300b067e2082\") " pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:28.700575 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700548 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3322fe7-5ad0-4f6d-9df7-300b067e2082-konnectivity-ca\") pod \"konnectivity-agent-dnw88\" (UID: \"b3322fe7-5ad0-4f6d-9df7-300b067e2082\") " pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:28.700575 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700573 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.700893 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700599 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-system-cni-dir\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.700893 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-cnibin\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.700893 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.700650 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.701200 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.701180 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.701920 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.701897 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 17:53:28.702055 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.702030 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:28.702124 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.702105 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lcbzt\"" Jul 12 17:53:28.702320 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.702302 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:28.703393 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.703368 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.704733 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.704530 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 17:53:28.705435 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.705066 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tvv4p\"" Jul 12 17:53:28.705435 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.705186 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:53:28.705929 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.705912 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.708201 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.707189 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 17:53:28.708201 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.707437 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-58ljb\"" Jul 12 17:53:28.708201 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.707716 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 17:53:28.708429 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.708416 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.709230 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.709175 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 17:53:28.709326 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.709278 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 17:53:28.709394 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.709372 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 17:53:28.709457 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.709397 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 17:53:28.709817 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.709645 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jhdfn\"" Jul 12 17:53:28.710855 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.710833 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 17:53:28.710950 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.710906 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:28.711011 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.710980 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:28.711456 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.711224 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 17:53:28.711456 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.711237 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w52gl\"" Jul 12 17:53:28.711456 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.711248 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 17:53:28.713701 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.713682 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.717011 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.716899 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 17:53:28.717110 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.717085 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zfktj\"" Jul 12 17:53:28.717488 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.717469 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 17:53:28.717781 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.717750 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 17:53:28.717875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.717845 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 17:53:28.717977 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.717959 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 17:53:28.718049 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.718042 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 17:53:28.736799 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.736305 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n7769" Jul 12 17:53:28.736799 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.736332 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n7769" Jul 12 17:53:28.790720 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.790698 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 17:53:28.800873 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.800843 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-host\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.801010 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.800893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-cnibin\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.801010 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.800932 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.801010 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.800968 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-cnibin\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.801184 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801027 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:28.801184 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801056 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d59c44a2-569d-4d7f-a56e-36129ded03a8-iptables-alerter-script\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.801184 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801153 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-device-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.801342 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801206 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.801342 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801237 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.801342 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801290 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-node-log\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801342 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801326 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovnkube-script-lib\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whsvf\" (UniqueName: \"kubernetes.io/projected/ecb81f47-62f9-4af4-91d2-cba45626d85d-kube-api-access-whsvf\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-run-netns\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801407 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.801549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1dec3aa-5c23-4a97-8827-90a57806b2b2-host\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.801549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801444 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1dec3aa-5c23-4a97-8827-90a57806b2b2-serviceca\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.801549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801470 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-cni-binary-copy\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.801549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801502 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3322fe7-5ad0-4f6d-9df7-300b067e2082-agent-certs\") pod \"konnectivity-agent-dnw88\" (UID: \"b3322fe7-5ad0-4f6d-9df7-300b067e2082\") " pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:28.801549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801601 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-etc-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovnkube-config\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-env-overrides\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801695 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fd52702-29ae-4863-b1a0-61ca7d08730e-tmp\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-kubernetes\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801722 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801729 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-system-cni-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801782 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-daemon-config\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-os-release\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801826 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-ovn\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801847 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysctl-d\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801886 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ckft\" (UniqueName: \"kubernetes.io/projected/9fd52702-29ae-4863-b1a0-61ca7d08730e-kube-api-access-7ckft\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801903 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-cni-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801926 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-kubelet\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801944 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801959 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-tuned\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.801996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.801989 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-os-release\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802015 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tdsp\" (UniqueName: \"kubernetes.io/projected/1b709f4a-171e-49cd-941f-f6856018b6a2-kube-api-access-8tdsp\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802044 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bvqm\" (UniqueName: \"kubernetes.io/projected/d59c44a2-569d-4d7f-a56e-36129ded03a8-kube-api-access-6bvqm\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802075 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-registration-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802080 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802100 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz72w\" (UniqueName: \"kubernetes.io/projected/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-kube-api-access-gz72w\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802117 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d59c44a2-569d-4d7f-a56e-36129ded03a8-host-slash\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802117 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-cni-binary-copy\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-modprobe-d\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802149 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysctl-conf\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802169 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-var-lib-kubelet\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-systemd\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.802219 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802313 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d907cd33-4123-4333-a38a-ac8e60b5ef04-kubelet-config\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802345 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-kubelet\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-socket-dir-parent\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802433 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d907cd33-4123-4333-a38a-ac8e60b5ef04-dbus\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.802913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802460 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b709f4a-171e-49cd-941f-f6856018b6a2-cni-binary-copy\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-cni-bin\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-hosts-file\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v26qw\" (UniqueName: \"kubernetes.io/projected/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-kube-api-access-v26qw\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802612 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-sys-fs\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-log-socket\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802667 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-run-ovn-kubernetes\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802702 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-cni-netd\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802727 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-cnibin\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802751 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-netns\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802777 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-cni-multus\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802834 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-systemd\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802859 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-sys\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802883 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-lib-modules\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.802911 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-hostroot\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.803823 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803063 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-multus-certs\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803094 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-var-lib-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803123 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-system-cni-dir\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803151 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-systemd-units\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-slash\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803219 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovn-node-metrics-cert\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803246 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-k8s-cni-cncf-io\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803271 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-socket-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803298 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57vtx\" (UniqueName: \"kubernetes.io/projected/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-kube-api-access-57vtx\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803332 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df4k7\" (UniqueName: \"kubernetes.io/projected/950b9259-0b9e-4847-838d-26aad602908c-kube-api-access-df4k7\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803364 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803393 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysconfig\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803419 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdwjc\" (UniqueName: \"kubernetes.io/projected/d1dec3aa-5c23-4a97-8827-90a57806b2b2-kube-api-access-gdwjc\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803466 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d907cd33-4123-4333-a38a-ac8e60b5ef04-dbus\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803488 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803517 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-run\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.804705 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-etc-kubernetes\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.803578 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-os-release\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.803715 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret podName:d907cd33-4123-4333-a38a-ac8e60b5ef04 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:29.30369183 +0000 UTC m=+2.947534549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret") pod "global-pull-secret-syncer-djndg" (UID: "d907cd33-4123-4333-a38a-ac8e60b5ef04") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.804031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d907cd33-4123-4333-a38a-ac8e60b5ef04-kubelet-config\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.804135 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/950b9259-0b9e-4847-838d-26aad602908c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.804151 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950b9259-0b9e-4847-838d-26aad602908c-system-cni-dir\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.804287 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-tmp-dir\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.804432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3322fe7-5ad0-4f6d-9df7-300b067e2082-konnectivity-ca\") pod \"konnectivity-agent-dnw88\" (UID: \"b3322fe7-5ad0-4f6d-9df7-300b067e2082\") " pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.804467 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-cni-bin\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.804496 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-conf-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.805514 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.805098 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3322fe7-5ad0-4f6d-9df7-300b067e2082-konnectivity-ca\") pod \"konnectivity-agent-dnw88\" (UID: \"b3322fe7-5ad0-4f6d-9df7-300b067e2082\") " pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:28.806632 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.806603 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3322fe7-5ad0-4f6d-9df7-300b067e2082-agent-certs\") pod \"konnectivity-agent-dnw88\" (UID: \"b3322fe7-5ad0-4f6d-9df7-300b067e2082\") " pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:28.812776 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.812752 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df4k7\" (UniqueName: \"kubernetes.io/projected/950b9259-0b9e-4847-838d-26aad602908c-kube-api-access-df4k7\") pod \"multus-additional-cni-plugins-wxmk8\" (UID: \"950b9259-0b9e-4847-838d-26aad602908c\") " pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:28.836160 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.836106 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" event={"ID":"73fc1951afb982df9c53cf26d8ff4718","Type":"ContainerStarted","Data":"780092d408826e6bfd4d38c20710c52c3b38925cfa99db3a26360da1df9ccea7"} Jul 12 17:53:28.837859 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.837829 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" event={"ID":"dc6ece70f95fe4adf85e0625be1d790d","Type":"ContainerStarted","Data":"c2748d61b58bab8a4396d1a118600e67f43d94ef19cbed746b650d4284022cda"} Jul 12 17:53:28.905821 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905791 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bvqm\" (UniqueName: \"kubernetes.io/projected/d59c44a2-569d-4d7f-a56e-36129ded03a8-kube-api-access-6bvqm\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.905821 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905828 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-registration-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz72w\" (UniqueName: \"kubernetes.io/projected/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-kube-api-access-gz72w\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d59c44a2-569d-4d7f-a56e-36129ded03a8-host-slash\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905897 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-modprobe-d\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysctl-conf\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-var-lib-kubelet\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905943 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d59c44a2-569d-4d7f-a56e-36129ded03a8-host-slash\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905941 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-registration-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.905963 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-systemd\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-kubelet\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906031 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-socket-dir-parent\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.906063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b709f4a-171e-49cd-941f-f6856018b6a2-cni-binary-copy\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906087 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-cni-bin\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906108 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysctl-conf\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-hosts-file\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906145 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v26qw\" (UniqueName: \"kubernetes.io/projected/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-kube-api-access-v26qw\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906170 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-sys-fs\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906207 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-kubelet\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906209 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-systemd\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906223 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-log-socket\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906260 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-socket-dir-parent\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-run-ovn-kubernetes\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906280 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-log-socket\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906059 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-modprobe-d\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906304 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-cni-netd\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-hosts-file\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-cnibin\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906364 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-run-ovn-kubernetes\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.906635 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906436 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906448 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-cnibin\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-netns\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906505 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-cni-multus\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906506 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-cni-netd\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-cni-bin\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906533 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-systemd\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906172 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-var-lib-kubelet\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906590 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-netns\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906605 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-cni-multus\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906595 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-sys-fs\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906647 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-systemd\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906650 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-sys\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906682 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-sys\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-lib-modules\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906745 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-hostroot\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906787 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-multus-certs\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906815 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-lib-modules\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.907494 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906823 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-var-lib-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906869 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-var-lib-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-systemd-units\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906872 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-multus-certs\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906908 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-systemd-units\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906907 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-hostroot\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906906 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-slash\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovn-node-metrics-cert\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.906992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-k8s-cni-cncf-io\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907019 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-slash\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-socket-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57vtx\" (UniqueName: \"kubernetes.io/projected/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-kube-api-access-57vtx\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907072 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-run-k8s-cni-cncf-io\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907191 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907219 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-socket-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907226 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysconfig\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907187 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b709f4a-171e-49cd-941f-f6856018b6a2-cni-binary-copy\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.908313 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysconfig\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907647 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907693 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdwjc\" (UniqueName: \"kubernetes.io/projected/d1dec3aa-5c23-4a97-8827-90a57806b2b2-kube-api-access-gdwjc\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.907962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908044 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-run\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908114 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-etc-kubernetes\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908148 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-tmp-dir\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908177 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-cni-bin\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908202 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-conf-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908230 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-host\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d59c44a2-569d-4d7f-a56e-36129ded03a8-iptables-alerter-script\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-device-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908410 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-device-dir\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908447 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-host\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908507 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-run\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.908572 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:28.909158 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.908682 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs podName:d6ee4bc0-6991-412a-884c-dbc5e5ef7665 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:29.408662617 +0000 UTC m=+3.052505335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs") pod "network-metrics-daemon-nxhsg" (UID: "d6ee4bc0-6991-412a-884c-dbc5e5ef7665") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908703 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-tmp-dir\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908785 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908806 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-etc-kubernetes\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-node-log\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-conf-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908898 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovnkube-script-lib\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908920 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-node-log\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908934 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-etc-selinux\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908961 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whsvf\" (UniqueName: \"kubernetes.io/projected/ecb81f47-62f9-4af4-91d2-cba45626d85d-kube-api-access-whsvf\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908983 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-cni-bin\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.908995 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-run-netns\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909024 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1dec3aa-5c23-4a97-8827-90a57806b2b2-host\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909051 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1dec3aa-5c23-4a97-8827-90a57806b2b2-serviceca\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909208 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-host-run-netns\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-etc-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909270 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovnkube-config\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909299 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-env-overrides\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909328 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fd52702-29ae-4863-b1a0-61ca7d08730e-tmp\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909384 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-etc-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-kubernetes\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.909502 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-system-cni-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910241 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-daemon-config\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910306 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-ovn\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910342 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysctl-d\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910377 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ckft\" (UniqueName: \"kubernetes.io/projected/9fd52702-29ae-4863-b1a0-61ca7d08730e-kube-api-access-7ckft\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910421 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-cni-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910456 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-kubelet\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910572 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-system-cni-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-ovn\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910640 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovnkube-script-lib\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910751 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.910934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910848 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-cni-dir\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.911608 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910996 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-sysctl-d\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.911608 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b709f4a-171e-49cd-941f-f6856018b6a2-multus-daemon-config\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.911608 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911510 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d1dec3aa-5c23-4a97-8827-90a57806b2b2-serviceca\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.911608 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911542 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-host-var-lib-kubelet\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.911608 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911602 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d1dec3aa-5c23-4a97-8827-90a57806b2b2-host\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.911907 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-kubernetes\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.911907 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.910490 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ecb81f47-62f9-4af4-91d2-cba45626d85d-run-openvswitch\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.911907 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d59c44a2-569d-4d7f-a56e-36129ded03a8-iptables-alerter-script\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.911907 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911818 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-tuned\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.911907 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911864 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-os-release\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.912149 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.911925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tdsp\" (UniqueName: \"kubernetes.io/projected/1b709f4a-171e-49cd-941f-f6856018b6a2-kube-api-access-8tdsp\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.912149 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.912015 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-env-overrides\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.912280 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.912244 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b709f4a-171e-49cd-941f-f6856018b6a2-os-release\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:28.912331 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.912295 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovnkube-config\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.913097 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.913063 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ecb81f47-62f9-4af4-91d2-cba45626d85d-ovn-node-metrics-cert\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.913737 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.913704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9fd52702-29ae-4863-b1a0-61ca7d08730e-tmp\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.916002 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.914859 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9fd52702-29ae-4863-b1a0-61ca7d08730e-etc-tuned\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.916403 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.916383 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:28.916481 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.916408 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:28.916481 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.916422 2568 projected.go:196] Error preparing data for projected volume kube-api-access-7k29d for pod openshift-network-diagnostics/network-check-target-cwgzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:28.916655 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:28.916502 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d podName:fde9c63f-8a31-4bb1-9170-d280505e5b97 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:29.416472652 +0000 UTC m=+3.060315373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7k29d" (UniqueName: "kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d") pod "network-check-target-cwgzb" (UID: "fde9c63f-8a31-4bb1-9170-d280505e5b97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:28.918709 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.918679 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz72w\" (UniqueName: \"kubernetes.io/projected/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-kube-api-access-gz72w\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:28.920185 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.920154 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdwjc\" (UniqueName: \"kubernetes.io/projected/d1dec3aa-5c23-4a97-8827-90a57806b2b2-kube-api-access-gdwjc\") pod \"node-ca-498ks\" (UID: \"d1dec3aa-5c23-4a97-8827-90a57806b2b2\") " pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:28.920321 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.920285 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ckft\" (UniqueName: \"kubernetes.io/projected/9fd52702-29ae-4863-b1a0-61ca7d08730e-kube-api-access-7ckft\") pod \"tuned-gt9wl\" (UID: \"9fd52702-29ae-4863-b1a0-61ca7d08730e\") " pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:28.921539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.921467 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v26qw\" (UniqueName: \"kubernetes.io/projected/bcaa496d-87e7-4d2f-a1de-2a67abf5cb68-kube-api-access-v26qw\") pod \"node-resolver-d7cld\" (UID: \"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68\") " pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:28.922132 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.922106 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whsvf\" (UniqueName: \"kubernetes.io/projected/ecb81f47-62f9-4af4-91d2-cba45626d85d-kube-api-access-whsvf\") pod \"ovnkube-node-4j476\" (UID: \"ecb81f47-62f9-4af4-91d2-cba45626d85d\") " pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:28.922465 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.922430 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bvqm\" (UniqueName: \"kubernetes.io/projected/d59c44a2-569d-4d7f-a56e-36129ded03a8-kube-api-access-6bvqm\") pod \"iptables-alerter-pwng9\" (UID: \"d59c44a2-569d-4d7f-a56e-36129ded03a8\") " pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:28.922753 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.922728 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57vtx\" (UniqueName: \"kubernetes.io/projected/52814fa4-f2cf-47cf-ab18-72b2e967dcf0-kube-api-access-57vtx\") pod \"aws-ebs-csi-driver-node-82g9k\" (UID: \"52814fa4-f2cf-47cf-ab18-72b2e967dcf0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:28.922923 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:28.922905 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tdsp\" (UniqueName: \"kubernetes.io/projected/1b709f4a-171e-49cd-941f-f6856018b6a2-kube-api-access-8tdsp\") pod \"multus-ww5j4\" (UID: \"1b709f4a-171e-49cd-941f-f6856018b6a2\") " pod="openshift-multus/multus-ww5j4" Jul 12 17:53:29.002034 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.001996 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:29.011062 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.011038 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" Jul 12 17:53:29.022627 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.022602 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ww5j4" Jul 12 17:53:29.036748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.036724 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pwng9" Jul 12 17:53:29.044728 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.044708 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" Jul 12 17:53:29.054654 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.054634 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-d7cld" Jul 12 17:53:29.061457 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.061434 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-498ks" Jul 12 17:53:29.070427 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.070400 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" Jul 12 17:53:29.076462 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.076441 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:29.113074 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.113043 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:29.316094 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.316002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:29.316258 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.316196 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:29.316336 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.316284 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret podName:d907cd33-4123-4333-a38a-ac8e60b5ef04 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:30.316261211 +0000 UTC m=+3.960103935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret") pod "global-pull-secret-syncer-djndg" (UID: "d907cd33-4123-4333-a38a-ac8e60b5ef04") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:29.417015 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.416966 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:29.417015 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.417017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:29.417255 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.417123 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:29.417255 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.417177 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:29.417255 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.417197 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:29.417255 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.417210 2568 projected.go:196] Error preparing data for projected volume kube-api-access-7k29d for pod openshift-network-diagnostics/network-check-target-cwgzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:29.417255 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.417204 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs podName:d6ee4bc0-6991-412a-884c-dbc5e5ef7665 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:30.417183923 +0000 UTC m=+4.061026635 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs") pod "network-metrics-daemon-nxhsg" (UID: "d6ee4bc0-6991-412a-884c-dbc5e5ef7665") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:29.417449 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.417269 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d podName:fde9c63f-8a31-4bb1-9170-d280505e5b97 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:30.417251158 +0000 UTC m=+4.061093882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7k29d" (UniqueName: "kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d") pod "network-check-target-cwgzb" (UID: "fde9c63f-8a31-4bb1-9170-d280505e5b97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:29.740040 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.739941 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 17:48:29 +0000 UTC" deadline="2028-04-25 06:52:30.251667964 +0000 UTC" Jul 12 17:53:29.740040 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.740002 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15660h59m0.511689205s" Jul 12 17:53:29.828910 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:29.828873 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:29.829072 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:29.828987 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:30.012607 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.012503 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3322fe7_5ad0_4f6d_9df7_300b067e2082.slice/crio-ddcd2df5a9f8f3d919f06c7ca4694478b45ae2fde703e8e5f334f5069f4393bd WatchSource:0}: Error finding container ddcd2df5a9f8f3d919f06c7ca4694478b45ae2fde703e8e5f334f5069f4393bd: Status 404 returned error can't find the container with id ddcd2df5a9f8f3d919f06c7ca4694478b45ae2fde703e8e5f334f5069f4393bd Jul 12 17:53:30.013929 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.013892 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd52702_29ae_4863_b1a0_61ca7d08730e.slice/crio-510df9ede72dfb12ec399edd6a1667b20474bb9faf1c3175f1e52b4043f2a7ec WatchSource:0}: Error finding container 510df9ede72dfb12ec399edd6a1667b20474bb9faf1c3175f1e52b4043f2a7ec: Status 404 returned error can't find the container with id 510df9ede72dfb12ec399edd6a1667b20474bb9faf1c3175f1e52b4043f2a7ec Jul 12 17:53:30.016268 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.016238 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59c44a2_569d_4d7f_a56e_36129ded03a8.slice/crio-8b297caa1b8ff79489806314a8b100cde04eb83da84ec953d933687244b5859a WatchSource:0}: Error finding container 8b297caa1b8ff79489806314a8b100cde04eb83da84ec953d933687244b5859a: Status 404 returned error can't find the container with id 8b297caa1b8ff79489806314a8b100cde04eb83da84ec953d933687244b5859a Jul 12 17:53:30.018397 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.018381 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1dec3aa_5c23_4a97_8827_90a57806b2b2.slice/crio-1f8d24c7a8fedbd50e6fca4242498422f339ddc65360215a71304756fba307e0 WatchSource:0}: Error finding container 1f8d24c7a8fedbd50e6fca4242498422f339ddc65360215a71304756fba307e0: Status 404 returned error can't find the container with id 1f8d24c7a8fedbd50e6fca4242498422f339ddc65360215a71304756fba307e0 Jul 12 17:53:30.020474 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.020453 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod950b9259_0b9e_4847_838d_26aad602908c.slice/crio-e417f0aabcec7516852d1021587ea0df2103eb4a2aab81e59e59c1a19727a665 WatchSource:0}: Error finding container e417f0aabcec7516852d1021587ea0df2103eb4a2aab81e59e59c1a19727a665: Status 404 returned error can't find the container with id e417f0aabcec7516852d1021587ea0df2103eb4a2aab81e59e59c1a19727a665 Jul 12 17:53:30.021072 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.020856 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcaa496d_87e7_4d2f_a1de_2a67abf5cb68.slice/crio-c300344c2c6813f93c38ddfbe3d1e5bd58779ad82899fde2a6740053eb5f31da WatchSource:0}: Error finding container c300344c2c6813f93c38ddfbe3d1e5bd58779ad82899fde2a6740053eb5f31da: Status 404 returned error can't find the container with id c300344c2c6813f93c38ddfbe3d1e5bd58779ad82899fde2a6740053eb5f31da Jul 12 17:53:30.022657 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.022564 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52814fa4_f2cf_47cf_ab18_72b2e967dcf0.slice/crio-c49804aefb36171f44fc0d132322e331fc5d73339f46d779b26bda8711aa8cec WatchSource:0}: Error finding container c49804aefb36171f44fc0d132322e331fc5d73339f46d779b26bda8711aa8cec: Status 404 returned error can't find the container with id c49804aefb36171f44fc0d132322e331fc5d73339f46d779b26bda8711aa8cec Jul 12 17:53:30.024290 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.024195 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb81f47_62f9_4af4_91d2_cba45626d85d.slice/crio-b07c62c3bcb3a8da6ca756f0300a256731b2690c6d830f4c9db513356c4f19fa WatchSource:0}: Error finding container b07c62c3bcb3a8da6ca756f0300a256731b2690c6d830f4c9db513356c4f19fa: Status 404 returned error can't find the container with id b07c62c3bcb3a8da6ca756f0300a256731b2690c6d830f4c9db513356c4f19fa Jul 12 17:53:30.025670 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:53:30.025540 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b709f4a_171e_49cd_941f_f6856018b6a2.slice/crio-6eeda9d9a0d90a348dcabbd9d9f136c9f5ac19601a10d25a8c40654bdb0606be WatchSource:0}: Error finding container 6eeda9d9a0d90a348dcabbd9d9f136c9f5ac19601a10d25a8c40654bdb0606be: Status 404 returned error can't find the container with id 6eeda9d9a0d90a348dcabbd9d9f136c9f5ac19601a10d25a8c40654bdb0606be Jul 12 17:53:30.323326 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.323082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:30.323473 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.323243 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:30.323473 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.323433 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret podName:d907cd33-4123-4333-a38a-ac8e60b5ef04 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:32.323416013 +0000 UTC m=+5.967258718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret") pod "global-pull-secret-syncer-djndg" (UID: "d907cd33-4123-4333-a38a-ac8e60b5ef04") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:30.424794 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.424741 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:30.424985 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.424802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:30.424985 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.424905 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:30.424985 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.424933 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:30.424985 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.424952 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:30.424985 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.424967 2568 projected.go:196] Error preparing data for projected volume kube-api-access-7k29d for pod openshift-network-diagnostics/network-check-target-cwgzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:30.425159 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.424991 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs podName:d6ee4bc0-6991-412a-884c-dbc5e5ef7665 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:32.424967832 +0000 UTC m=+6.068810548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs") pod "network-metrics-daemon-nxhsg" (UID: "d6ee4bc0-6991-412a-884c-dbc5e5ef7665") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:30.425159 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.425011 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d podName:fde9c63f-8a31-4bb1-9170-d280505e5b97 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:32.424998984 +0000 UTC m=+6.068841687 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7k29d" (UniqueName: "kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d") pod "network-check-target-cwgzb" (UID: "fde9c63f-8a31-4bb1-9170-d280505e5b97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:30.599284 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.599022 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 17:53:30.740680 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.740585 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 17:48:29 +0000 UTC" deadline="2028-03-24 23:40:15.629954326 +0000 UTC" Jul 12 17:53:30.740680 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.740626 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14909h46m44.889333909s" Jul 12 17:53:30.832207 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.831452 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:30.832207 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.831609 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:30.832207 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.832039 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:30.832207 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:30.832152 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:30.860293 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.860204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" event={"ID":"73fc1951afb982df9c53cf26d8ff4718","Type":"ContainerStarted","Data":"21db4557c8e168638dcd26ae8f7ad03cb104c89f4af5a5579257fd80b5611a88"} Jul 12 17:53:30.869524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.869486 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" event={"ID":"950b9259-0b9e-4847-838d-26aad602908c","Type":"ContainerStarted","Data":"e417f0aabcec7516852d1021587ea0df2103eb4a2aab81e59e59c1a19727a665"} Jul 12 17:53:30.878066 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.878000 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-136.ec2.internal" podStartSLOduration=2.87798046 podStartE2EDuration="2.87798046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:30.877465657 +0000 UTC m=+4.521308384" watchObservedRunningTime="2026-07-12 17:53:30.87798046 +0000 UTC m=+4.521823188" Jul 12 17:53:30.881596 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.881496 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ww5j4" event={"ID":"1b709f4a-171e-49cd-941f-f6856018b6a2","Type":"ContainerStarted","Data":"6eeda9d9a0d90a348dcabbd9d9f136c9f5ac19601a10d25a8c40654bdb0606be"} Jul 12 17:53:30.884543 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.884512 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" event={"ID":"52814fa4-f2cf-47cf-ab18-72b2e967dcf0","Type":"ContainerStarted","Data":"c49804aefb36171f44fc0d132322e331fc5d73339f46d779b26bda8711aa8cec"} Jul 12 17:53:30.898510 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.898474 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pwng9" event={"ID":"d59c44a2-569d-4d7f-a56e-36129ded03a8","Type":"ContainerStarted","Data":"8b297caa1b8ff79489806314a8b100cde04eb83da84ec953d933687244b5859a"} Jul 12 17:53:30.907893 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.907861 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dnw88" event={"ID":"b3322fe7-5ad0-4f6d-9df7-300b067e2082","Type":"ContainerStarted","Data":"ddcd2df5a9f8f3d919f06c7ca4694478b45ae2fde703e8e5f334f5069f4393bd"} Jul 12 17:53:30.913222 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.913162 2568 generic.go:356] "Generic (PLEG): container finished" podID="dc6ece70f95fe4adf85e0625be1d790d" containerID="30e18119e4ebf57387524ca12675347529e78bd0bddf40ce34043f3c10a07472" exitCode=0 Jul 12 17:53:30.913363 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.913251 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" event={"ID":"dc6ece70f95fe4adf85e0625be1d790d","Type":"ContainerDied","Data":"30e18119e4ebf57387524ca12675347529e78bd0bddf40ce34043f3c10a07472"} Jul 12 17:53:30.919952 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.919920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"b07c62c3bcb3a8da6ca756f0300a256731b2690c6d830f4c9db513356c4f19fa"} Jul 12 17:53:30.922209 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.922166 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d7cld" event={"ID":"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68","Type":"ContainerStarted","Data":"c300344c2c6813f93c38ddfbe3d1e5bd58779ad82899fde2a6740053eb5f31da"} Jul 12 17:53:30.924597 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.924557 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-498ks" event={"ID":"d1dec3aa-5c23-4a97-8827-90a57806b2b2","Type":"ContainerStarted","Data":"1f8d24c7a8fedbd50e6fca4242498422f339ddc65360215a71304756fba307e0"} Jul 12 17:53:30.934242 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:30.934179 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" event={"ID":"9fd52702-29ae-4863-b1a0-61ca7d08730e","Type":"ContainerStarted","Data":"510df9ede72dfb12ec399edd6a1667b20474bb9faf1c3175f1e52b4043f2a7ec"} Jul 12 17:53:31.829608 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:31.829567 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:31.830092 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:31.829724 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:31.946677 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:31.946635 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" event={"ID":"dc6ece70f95fe4adf85e0625be1d790d","Type":"ContainerStarted","Data":"c7fdf8dfae9640f815fea3bef586323b287616821b646d5ad14b5a3b6db4ec9d"} Jul 12 17:53:31.963954 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:31.963892 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-136.ec2.internal" podStartSLOduration=3.963870633 podStartE2EDuration="3.963870633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:31.96286234 +0000 UTC m=+5.606705062" watchObservedRunningTime="2026-07-12 17:53:31.963870633 +0000 UTC m=+5.607713361" Jul 12 17:53:32.340420 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:32.340381 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:32.340677 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.340605 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:32.340748 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.340684 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret podName:d907cd33-4123-4333-a38a-ac8e60b5ef04 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:36.340661305 +0000 UTC m=+9.984504025 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret") pod "global-pull-secret-syncer-djndg" (UID: "d907cd33-4123-4333-a38a-ac8e60b5ef04") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:32.441193 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:32.441149 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:32.441372 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:32.441268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:32.441450 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.441428 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:32.441505 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.441499 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs podName:d6ee4bc0-6991-412a-884c-dbc5e5ef7665 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:36.441476865 +0000 UTC m=+10.085319589 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs") pod "network-metrics-daemon-nxhsg" (UID: "d6ee4bc0-6991-412a-884c-dbc5e5ef7665") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:32.441987 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.441960 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:32.441987 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.441989 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:32.442153 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.442002 2568 projected.go:196] Error preparing data for projected volume kube-api-access-7k29d for pod openshift-network-diagnostics/network-check-target-cwgzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:32.442153 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.442053 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d podName:fde9c63f-8a31-4bb1-9170-d280505e5b97 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:36.442036919 +0000 UTC m=+10.085879625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7k29d" (UniqueName: "kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d") pod "network-check-target-cwgzb" (UID: "fde9c63f-8a31-4bb1-9170-d280505e5b97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:32.832128 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:32.831264 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:32.832128 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.831387 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:32.832128 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:32.831531 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:32.832128 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:32.831713 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:33.829091 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:33.829060 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:33.829260 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:33.829202 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:34.829172 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:34.829084 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:34.829609 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:34.829226 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:34.829665 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:34.829623 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:34.829753 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:34.829728 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:35.829694 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:35.829631 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:35.830136 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:35.829768 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:36.380668 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:36.380624 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:36.380866 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.380816 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:36.380946 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.380877 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret podName:d907cd33-4123-4333-a38a-ac8e60b5ef04 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:44.380858279 +0000 UTC m=+18.024700984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret") pod "global-pull-secret-syncer-djndg" (UID: "d907cd33-4123-4333-a38a-ac8e60b5ef04") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:36.481258 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:36.481217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:36.481444 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:36.481279 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:36.481444 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.481401 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:36.481444 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.481437 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:36.481616 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.481454 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:36.481616 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.481467 2568 projected.go:196] Error preparing data for projected volume kube-api-access-7k29d for pod openshift-network-diagnostics/network-check-target-cwgzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:36.481616 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.481478 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs podName:d6ee4bc0-6991-412a-884c-dbc5e5ef7665 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:44.481457503 +0000 UTC m=+18.125300205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs") pod "network-metrics-daemon-nxhsg" (UID: "d6ee4bc0-6991-412a-884c-dbc5e5ef7665") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:36.481616 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.481526 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d podName:fde9c63f-8a31-4bb1-9170-d280505e5b97 nodeName:}" failed. No retries permitted until 2026-07-12 17:53:44.481499536 +0000 UTC m=+18.125342245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7k29d" (UniqueName: "kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d") pod "network-check-target-cwgzb" (UID: "fde9c63f-8a31-4bb1-9170-d280505e5b97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:36.831265 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:36.830597 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:36.831265 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.830721 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:36.831265 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:36.831096 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:36.831265 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:36.831180 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:37.829784 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:37.829743 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:37.829970 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:37.829871 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:38.830639 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:38.830607 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:38.831130 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:38.830756 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:38.831130 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:38.830803 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:38.831130 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:38.830934 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:39.829467 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:39.829430 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:39.829659 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:39.829601 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:40.829046 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:40.828961 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:40.829473 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:40.828964 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:40.829473 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:40.829084 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:40.829473 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:40.829149 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:41.829090 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:41.829057 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:41.829560 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:41.829185 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:42.829093 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:42.829054 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:42.829529 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:42.829115 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:42.829529 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:42.829204 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:42.829529 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:42.829320 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:43.829212 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:43.829176 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:43.829783 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:43.829300 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:44.445843 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:44.445805 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:44.446000 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.445967 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:44.446048 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.446035 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret podName:d907cd33-4123-4333-a38a-ac8e60b5ef04 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:00.446016518 +0000 UTC m=+34.089859220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret") pod "global-pull-secret-syncer-djndg" (UID: "d907cd33-4123-4333-a38a-ac8e60b5ef04") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:53:44.547094 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:44.547017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:44.547094 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:44.547059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:44.547292 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.547180 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:44.547292 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.547208 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:53:44.547292 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.547225 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:53:44.547292 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.547237 2568 projected.go:196] Error preparing data for projected volume kube-api-access-7k29d for pod openshift-network-diagnostics/network-check-target-cwgzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:44.547292 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.547257 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs podName:d6ee4bc0-6991-412a-884c-dbc5e5ef7665 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:00.547237519 +0000 UTC m=+34.191080225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs") pod "network-metrics-daemon-nxhsg" (UID: "d6ee4bc0-6991-412a-884c-dbc5e5ef7665") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:53:44.547292 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.547281 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d podName:fde9c63f-8a31-4bb1-9170-d280505e5b97 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:00.547270224 +0000 UTC m=+34.191112926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7k29d" (UniqueName: "kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d") pod "network-check-target-cwgzb" (UID: "fde9c63f-8a31-4bb1-9170-d280505e5b97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:53:44.828908 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:44.828829 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:44.829073 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:44.829007 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:44.829138 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.828999 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:44.829198 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:44.829128 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:45.829858 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:45.829822 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:45.830320 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:45.829966 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:46.829683 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:46.829642 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:46.829867 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:46.829763 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:46.829867 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:46.829842 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:46.830258 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:46.829927 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:47.829832 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:47.829796 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:47.830015 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:47.829913 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:48.829765 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.829444 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:48.829872 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.829497 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:48.829925 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:48.829889 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:48.829981 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:48.829954 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:48.986030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.985782 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"cffa71478ad83e929641acf7a5955ac5a1acbe4e9fde3d8bc8b889632232c9a1"} Jul 12 17:53:48.986807 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.986101 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"87af483b55fe060ba6a8dd0f628feb0152bcd1dd398695a5604a7b56c35bff9b"} Jul 12 17:53:48.987318 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.987291 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-d7cld" event={"ID":"bcaa496d-87e7-4d2f-a1de-2a67abf5cb68","Type":"ContainerStarted","Data":"f9a77dca2595e71cdfdbc325d72bff52ab2edfbeb0e91a4917836eca22ec0b5c"} Jul 12 17:53:48.989002 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.988971 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-498ks" event={"ID":"d1dec3aa-5c23-4a97-8827-90a57806b2b2","Type":"ContainerStarted","Data":"bc1381b2598d3e870e85d03d70a9c2e2912f8475d81de1cc3e84b6eaed94f855"} Jul 12 17:53:48.990611 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.990584 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" event={"ID":"9fd52702-29ae-4863-b1a0-61ca7d08730e","Type":"ContainerStarted","Data":"08256218750a3917f9f646dabde69c77228d5eaa816c5bdaaf3b2ae0a534ae0d"} Jul 12 17:53:48.992657 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.992624 2568 generic.go:356] "Generic (PLEG): container finished" podID="950b9259-0b9e-4847-838d-26aad602908c" containerID="9772adbd31d7dc7d6fd943e24f9f2e1ad7e79f47ff15df5b0ed86369addd1be4" exitCode=0 Jul 12 17:53:48.992749 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.992681 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" event={"ID":"950b9259-0b9e-4847-838d-26aad602908c","Type":"ContainerDied","Data":"9772adbd31d7dc7d6fd943e24f9f2e1ad7e79f47ff15df5b0ed86369addd1be4"} Jul 12 17:53:48.994296 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.994274 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ww5j4" event={"ID":"1b709f4a-171e-49cd-941f-f6856018b6a2","Type":"ContainerStarted","Data":"a3f2b486f23093f6e9787a82e2774b5926574a46809c24974b7ccd272227b78d"} Jul 12 17:53:48.995815 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.995792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" event={"ID":"52814fa4-f2cf-47cf-ab18-72b2e967dcf0","Type":"ContainerStarted","Data":"daeb36d963502bfcdd896dcee0b350afbcc9aaf9d6affa911b9afc6e7c92404e"} Jul 12 17:53:48.997141 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:48.997115 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dnw88" event={"ID":"b3322fe7-5ad0-4f6d-9df7-300b067e2082","Type":"ContainerStarted","Data":"7ca9f2485363589c266814a6b8fff87ea5be3bc0b75351eec30e80beb392c5d7"} Jul 12 17:53:49.003718 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.003672 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-d7cld" podStartSLOduration=3.404112811 podStartE2EDuration="22.003654489s" totalImagesPullingTime="18.599541678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:49.003030984 +0000 UTC m=+22.646873708" watchObservedRunningTime="2026-07-12 17:53:49.003654489 +0000 UTC m=+22.647497213" Jul 12 17:53:49.029378 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.029342 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-498ks" podStartSLOduration=3.426589494 podStartE2EDuration="22.029328661s" totalImagesPullingTime="18.602739167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:49.016036389 +0000 UTC m=+22.659879115" watchObservedRunningTime="2026-07-12 17:53:49.029328661 +0000 UTC m=+22.673171383" Jul 12 17:53:49.029840 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.029818 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dnw88" podStartSLOduration=7.790809625 podStartE2EDuration="22.029809858s" totalImagesPullingTime="14.239000233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:49.029214717 +0000 UTC m=+22.673057439" watchObservedRunningTime="2026-07-12 17:53:49.029809858 +0000 UTC m=+22.673652581" Jul 12 17:53:49.063377 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.063336 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gt9wl" podStartSLOduration=3.454354377 podStartE2EDuration="22.063318772s" totalImagesPullingTime="18.608964395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:49.062782203 +0000 UTC m=+22.706624926" watchObservedRunningTime="2026-07-12 17:53:49.063318772 +0000 UTC m=+22.707161494" Jul 12 17:53:49.079801 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.079758 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ww5j4" podStartSLOduration=3.434583751 podStartE2EDuration="22.079744287s" totalImagesPullingTime="18.645160536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:49.079460102 +0000 UTC m=+22.723302822" watchObservedRunningTime="2026-07-12 17:53:49.079744287 +0000 UTC m=+22.723587009" Jul 12 17:53:49.198480 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.198454 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:49.252174 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.252133 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:49.353012 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.352979 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:49.546221 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.546074 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:49.612768 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.612728 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:49.665825 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.665791 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:49.829766 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:49.829694 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:49.829972 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:49.829821 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:50.001199 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.000985 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pwng9" event={"ID":"d59c44a2-569d-4d7f-a56e-36129ded03a8","Type":"ContainerStarted","Data":"7086f6fd560c0c6398f4e903c4beb7c0b1a98924c8f9acfed37e8a7a03aa9c6b"} Jul 12 17:53:50.005420 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.004873 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"1470c682450b0f4dd705d8c623258fd9e33aaa2b296f5582eab1111258778771"} Jul 12 17:53:50.005420 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.004952 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"6735fcfcbf94e38492aa427f4219b3caec28bf105592a123f700811e95119b2f"} Jul 12 17:53:50.005420 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.005003 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"1ad2d296b6631b43d6c7cb873a4eb0ffaef3858b08cd50eb51024c49aca22678"} Jul 12 17:53:50.005420 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.005050 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"dedf38c7b30e15045c3f2d7166ae8e9339a720d5a6f63318b8824536360a740d"} Jul 12 17:53:50.011859 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.011814 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:50.019983 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.019944 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pwng9" podStartSLOduration=12.469031574 podStartE2EDuration="23.019928065s" totalImagesPullingTime="10.550896491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:50.019607299 +0000 UTC m=+23.663450021" watchObservedRunningTime="2026-07-12 17:53:50.019928065 +0000 UTC m=+23.663770765" Jul 12 17:53:50.092001 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.091969 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 17:53:50.111914 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.111886 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:50.765396 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.765214 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T17:53:50.091989939Z","UUID":"42a2d53c-3dd0-4227-b114-43b070743337","Handler":null,"Name":"","Endpoint":""} Jul 12 17:53:50.766988 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.766965 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 17:53:50.767121 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.767023 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 17:53:50.824780 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.824741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:50.829814 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.829787 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:50.829971 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:50.829799 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:50.829971 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:50.829911 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:50.830080 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:50.829975 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:51.008644 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:51.008597 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" event={"ID":"52814fa4-f2cf-47cf-ab18-72b2e967dcf0","Type":"ContainerStarted","Data":"26ea7d9f71cd81e0f291b72d5e47efc1381aab55ae06b71402dd7c44fc4907e2"} Jul 12 17:53:51.220113 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:51.220079 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:51.321401 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:51.321209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:51.829654 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:51.829617 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:51.829903 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:51.829738 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:52.016228 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.016196 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"ff41e6c798b686748dc1475cc342ee62292dbc0b9f1b05cd9cdd18c12ab2e4f8"} Jul 12 17:53:52.016777 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.016433 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:52.018492 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.018462 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" event={"ID":"52814fa4-f2cf-47cf-ab18-72b2e967dcf0","Type":"ContainerStarted","Data":"56d5f671d638a20c06df8f2eb4b204aef12bc2801acbe3218a907456ad6b3fa9"} Jul 12 17:53:52.036447 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.036401 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-82g9k" podStartSLOduration=4.117119238 podStartE2EDuration="25.036386674s" totalImagesPullingTime="20.919267436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:52.035770729 +0000 UTC m=+25.679613451" watchObservedRunningTime="2026-07-12 17:53:52.036386674 +0000 UTC m=+25.680229396" Jul 12 17:53:52.121925 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.121828 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:52.364500 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.364471 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:52.466268 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.466233 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:52.566702 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.566674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:52.829486 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.829408 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:52.829486 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:52.829444 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:52.829711 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:52.829578 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:52.829711 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:52.829684 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:53.223967 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:53.223934 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:53.224477 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:53.224463 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:53.497002 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:53.496978 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:53.497406 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:53.497390 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dnw88" Jul 12 17:53:53.552732 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:53.552706 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:53:53.829187 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:53.828946 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:53.829187 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:53.829182 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:54.026683 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:54.026646 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" event={"ID":"ecb81f47-62f9-4af4-91d2-cba45626d85d","Type":"ContainerStarted","Data":"e020ba7dba5abd3c66870fbee69b5bfe78ac8a0302acb662810537eb9a62a72f"} Jul 12 17:53:54.027022 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:54.026972 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:54.028491 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:54.028470 2568 generic.go:356] "Generic (PLEG): container finished" podID="950b9259-0b9e-4847-838d-26aad602908c" containerID="acdcd01592f6f8538fd38ffcca288b80972c495599dbd893b4602bce99d6a3a0" exitCode=0 Jul 12 17:53:54.028626 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:54.028542 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" event={"ID":"950b9259-0b9e-4847-838d-26aad602908c","Type":"ContainerDied","Data":"acdcd01592f6f8538fd38ffcca288b80972c495599dbd893b4602bce99d6a3a0"} Jul 12 17:53:54.042769 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:54.042748 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:54.051273 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:54.051232 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" podStartSLOduration=8.377842984 podStartE2EDuration="27.051219594s" totalImagesPullingTime="18.67337661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:53:54.050951225 +0000 UTC m=+27.694793947" watchObservedRunningTime="2026-07-12 17:53:54.051219594 +0000 UTC m=+27.695062317" Jul 12 17:53:54.833897 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:54.833861 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:54.834771 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:54.834003 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:54.834771 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:54.834372 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:54.834771 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:54.834454 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:55.031481 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.031447 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:55.031678 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.031551 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:55.046894 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.046869 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:53:55.495550 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.495359 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-djndg"] Jul 12 17:53:55.495706 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.495666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:55.495770 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:55.495751 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:55.498470 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.498440 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nxhsg"] Jul 12 17:53:55.498610 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.498586 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:55.498716 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:55.498696 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:55.499199 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.499177 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cwgzb"] Jul 12 17:53:55.499284 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:55.499271 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:55.499379 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:55.499359 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:56.035357 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:56.035324 2568 generic.go:356] "Generic (PLEG): container finished" podID="950b9259-0b9e-4847-838d-26aad602908c" containerID="2467a1c1349bb52b48e8c06cd577d19bc648401de2cb1500d1b3c2bd0a835524" exitCode=0 Jul 12 17:53:56.035802 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:56.035389 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" event={"ID":"950b9259-0b9e-4847-838d-26aad602908c","Type":"ContainerDied","Data":"2467a1c1349bb52b48e8c06cd577d19bc648401de2cb1500d1b3c2bd0a835524"} Jul 12 17:53:57.829720 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:57.829632 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:57.829720 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:57.829649 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:57.830426 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:57.829759 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:53:57.830426 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:57.829651 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:57.830426 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:57.829803 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:57.830426 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:57.829864 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:58.041957 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:58.041917 2568 generic.go:356] "Generic (PLEG): container finished" podID="950b9259-0b9e-4847-838d-26aad602908c" containerID="4cfc56f934c5103be661cb0fda12574492a60ae3635d106c4ad7b32997a33664" exitCode=0 Jul 12 17:53:58.042130 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:58.041973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" event={"ID":"950b9259-0b9e-4847-838d-26aad602908c","Type":"ContainerDied","Data":"4cfc56f934c5103be661cb0fda12574492a60ae3635d106c4ad7b32997a33664"} Jul 12 17:53:59.829220 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:59.829142 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:53:59.829836 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:59.829148 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:53:59.829836 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:59.829259 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cwgzb" podUID="fde9c63f-8a31-4bb1-9170-d280505e5b97" Jul 12 17:53:59.829836 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:59.829342 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-djndg" podUID="d907cd33-4123-4333-a38a-ac8e60b5ef04" Jul 12 17:53:59.829836 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:53:59.829148 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:53:59.829836 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:53:59.829451 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nxhsg" podUID="d6ee4bc0-6991-412a-884c-dbc5e5ef7665" Jul 12 17:54:00.473262 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:00.473226 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:54:00.473455 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:00.473402 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 17:54:00.473547 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:00.473481 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret podName:d907cd33-4123-4333-a38a-ac8e60b5ef04 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:32.473462365 +0000 UTC m=+66.117305072 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret") pod "global-pull-secret-syncer-djndg" (UID: "d907cd33-4123-4333-a38a-ac8e60b5ef04") : object "kube-system"/"original-pull-secret" not registered Jul 12 17:54:00.574346 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:00.574314 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:54:00.574346 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:00.574352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:54:00.574618 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:00.574481 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:54:00.574618 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:00.574574 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs podName:d6ee4bc0-6991-412a-884c-dbc5e5ef7665 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:32.574551836 +0000 UTC m=+66.218394551 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs") pod "network-metrics-daemon-nxhsg" (UID: "d6ee4bc0-6991-412a-884c-dbc5e5ef7665") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 17:54:00.574618 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:00.574487 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 17:54:00.574618 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:00.574608 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 17:54:00.574618 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:00.574621 2568 projected.go:196] Error preparing data for projected volume kube-api-access-7k29d for pod openshift-network-diagnostics/network-check-target-cwgzb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:54:00.574873 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:00.574677 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d podName:fde9c63f-8a31-4bb1-9170-d280505e5b97 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:32.574661202 +0000 UTC m=+66.218503916 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7k29d" (UniqueName: "kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d") pod "network-check-target-cwgzb" (UID: "fde9c63f-8a31-4bb1-9170-d280505e5b97") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 17:54:01.395784 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.395744 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-136.ec2.internal" event="NodeReady" Jul 12 17:54:01.396217 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.395928 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 17:54:01.428820 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.428788 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b958f7c4c-wlvg6"] Jul 12 17:54:01.456257 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.456220 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jvjpf"] Jul 12 17:54:01.456433 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.456356 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.458995 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.458972 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 17:54:01.459133 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.459009 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qczrr\"" Jul 12 17:54:01.459133 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.459008 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 17:54:01.459133 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.459079 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 17:54:01.464018 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.463999 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 17:54:01.472277 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.472255 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jvjpf" Jul 12 17:54:01.472382 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.472254 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nzstl"] Jul 12 17:54:01.475059 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.475039 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-59fr9\"" Jul 12 17:54:01.475160 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.475079 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 17:54:01.475160 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.475103 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 17:54:01.475380 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.475361 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 17:54:01.490248 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.490220 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b958f7c4c-wlvg6"] Jul 12 17:54:01.490414 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.490259 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.490414 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.490291 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jvjpf"] Jul 12 17:54:01.490414 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.490320 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nzstl"] Jul 12 17:54:01.492941 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.492919 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 17:54:01.493057 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.492993 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 17:54:01.493172 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.493150 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xtknx\"" Jul 12 17:54:01.567392 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.567357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:01.582239 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-installation-pull-secrets\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.582416 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582243 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68db3a53-54e6-4a52-9b0b-63f9aedb0c83-cert\") pod \"ingress-canary-jvjpf\" (UID: \"68db3a53-54e6-4a52-9b0b-63f9aedb0c83\") " pod="openshift-ingress-canary/ingress-canary-jvjpf" Jul 12 17:54:01.582416 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582279 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-image-registry-private-configuration\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.582416 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.582416 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-bound-sa-token\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.582644 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582423 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cee3c823-d6f7-4498-b550-bd04f28d6152-tmp-dir\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.582644 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/731149e1-0bb8-440e-a36f-a7f65052bf23-ca-trust-extracted\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.582644 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582550 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cee3c823-d6f7-4498-b550-bd04f28d6152-config-volume\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.582644 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582575 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llm8d\" (UniqueName: \"kubernetes.io/projected/cee3c823-d6f7-4498-b550-bd04f28d6152-kube-api-access-llm8d\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.582644 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582601 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-certificates\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.582838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582648 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.582838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582667 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-trusted-ca\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.582838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582701 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56rr2\" (UniqueName: \"kubernetes.io/projected/68db3a53-54e6-4a52-9b0b-63f9aedb0c83-kube-api-access-56rr2\") pod \"ingress-canary-jvjpf\" (UID: \"68db3a53-54e6-4a52-9b0b-63f9aedb0c83\") " pod="openshift-ingress-canary/ingress-canary-jvjpf" Jul 12 17:54:01.582838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.582717 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgdbq\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-kube-api-access-jgdbq\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684286 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-image-registry-private-configuration\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684286 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.684534 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684309 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-bound-sa-token\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684534 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684333 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cee3c823-d6f7-4498-b550-bd04f28d6152-tmp-dir\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.684534 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684379 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/731149e1-0bb8-440e-a36f-a7f65052bf23-ca-trust-extracted\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684534 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684400 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cee3c823-d6f7-4498-b550-bd04f28d6152-config-volume\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.684534 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llm8d\" (UniqueName: \"kubernetes.io/projected/cee3c823-d6f7-4498-b550-bd04f28d6152-kube-api-access-llm8d\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.684534 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:01.684440 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:54:01.684534 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:01.684525 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls podName:cee3c823-d6f7-4498-b550-bd04f28d6152 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:02.184489704 +0000 UTC m=+35.828332419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls") pod "dns-default-nzstl" (UID: "cee3c823-d6f7-4498-b550-bd04f28d6152") : secret "dns-default-metrics-tls" not found Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684446 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-certificates\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-trusted-ca\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684724 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cee3c823-d6f7-4498-b550-bd04f28d6152-tmp-dir\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684737 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56rr2\" (UniqueName: \"kubernetes.io/projected/68db3a53-54e6-4a52-9b0b-63f9aedb0c83-kube-api-access-56rr2\") pod \"ingress-canary-jvjpf\" (UID: \"68db3a53-54e6-4a52-9b0b-63f9aedb0c83\") " pod="openshift-ingress-canary/ingress-canary-jvjpf" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684784 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgdbq\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-kube-api-access-jgdbq\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:01.684816 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:01.684828 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b958f7c4c-wlvg6: secret "image-registry-tls" not found Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684830 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-installation-pull-secrets\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684839 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/731149e1-0bb8-440e-a36f-a7f65052bf23-ca-trust-extracted\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.684848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68db3a53-54e6-4a52-9b0b-63f9aedb0c83-cert\") pod \"ingress-canary-jvjpf\" (UID: \"68db3a53-54e6-4a52-9b0b-63f9aedb0c83\") " pod="openshift-ingress-canary/ingress-canary-jvjpf" Jul 12 17:54:01.684875 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:01.684866 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls podName:731149e1-0bb8-440e-a36f-a7f65052bf23 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:02.184853293 +0000 UTC m=+35.828696018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls") pod "image-registry-7b958f7c4c-wlvg6" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23") : secret "image-registry-tls" not found Jul 12 17:54:01.685435 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.685076 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cee3c823-d6f7-4498-b550-bd04f28d6152-config-volume\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.685611 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.685498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-certificates\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.686251 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.686186 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-trusted-ca\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.689035 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.689012 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68db3a53-54e6-4a52-9b0b-63f9aedb0c83-cert\") pod \"ingress-canary-jvjpf\" (UID: \"68db3a53-54e6-4a52-9b0b-63f9aedb0c83\") " pod="openshift-ingress-canary/ingress-canary-jvjpf" Jul 12 17:54:01.689143 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.689015 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-installation-pull-secrets\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.689143 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.689021 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-image-registry-private-configuration\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.693283 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.693241 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56rr2\" (UniqueName: \"kubernetes.io/projected/68db3a53-54e6-4a52-9b0b-63f9aedb0c83-kube-api-access-56rr2\") pod \"ingress-canary-jvjpf\" (UID: \"68db3a53-54e6-4a52-9b0b-63f9aedb0c83\") " pod="openshift-ingress-canary/ingress-canary-jvjpf" Jul 12 17:54:01.693542 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.693479 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llm8d\" (UniqueName: \"kubernetes.io/projected/cee3c823-d6f7-4498-b550-bd04f28d6152-kube-api-access-llm8d\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:01.693659 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.693640 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-bound-sa-token\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.694157 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.694137 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgdbq\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-kube-api-access-jgdbq\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:01.783015 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.782979 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jvjpf" Jul 12 17:54:01.829158 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.829129 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:54:01.829337 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.829136 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:54:01.829419 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.829147 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:54:01.832585 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.832564 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 17:54:01.832727 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.832671 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9zfm\"" Jul 12 17:54:01.832981 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.832961 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-84rf9\"" Jul 12 17:54:01.833088 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.832987 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 17:54:01.833088 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.833006 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 17:54:01.833179 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:01.833123 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 17:54:02.190445 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:02.190405 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:02.190662 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:02.190488 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:02.190662 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:02.190619 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:54:02.190662 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:02.190643 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b958f7c4c-wlvg6: secret "image-registry-tls" not found Jul 12 17:54:02.190803 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:02.190663 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:54:02.190803 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:02.190709 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls podName:731149e1-0bb8-440e-a36f-a7f65052bf23 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:03.190687491 +0000 UTC m=+36.834530215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls") pod "image-registry-7b958f7c4c-wlvg6" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23") : secret "image-registry-tls" not found Jul 12 17:54:02.190803 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:02.190728 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls podName:cee3c823-d6f7-4498-b550-bd04f28d6152 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:03.190719555 +0000 UTC m=+36.834562258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls") pod "dns-default-nzstl" (UID: "cee3c823-d6f7-4498-b550-bd04f28d6152") : secret "dns-default-metrics-tls" not found Jul 12 17:54:02.536370 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:02.536339 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jvjpf"] Jul 12 17:54:02.556454 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:02.556422 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68db3a53_54e6_4a52_9b0b_63f9aedb0c83.slice/crio-f1e003d657f0f95e0f6af17a8a33a34897015469605623d19e23b2d167a5c67b WatchSource:0}: Error finding container f1e003d657f0f95e0f6af17a8a33a34897015469605623d19e23b2d167a5c67b: Status 404 returned error can't find the container with id f1e003d657f0f95e0f6af17a8a33a34897015469605623d19e23b2d167a5c67b Jul 12 17:54:03.058214 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.058139 2568 generic.go:356] "Generic (PLEG): container finished" podID="950b9259-0b9e-4847-838d-26aad602908c" containerID="12fd8db9217bdc765cecc04d17bde69506ad18118d4be2badc76dd9ba6e79cbd" exitCode=0 Jul 12 17:54:03.058369 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.058223 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" event={"ID":"950b9259-0b9e-4847-838d-26aad602908c","Type":"ContainerDied","Data":"12fd8db9217bdc765cecc04d17bde69506ad18118d4be2badc76dd9ba6e79cbd"} Jul 12 17:54:03.059423 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.059389 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jvjpf" event={"ID":"68db3a53-54e6-4a52-9b0b-63f9aedb0c83","Type":"ContainerStarted","Data":"f1e003d657f0f95e0f6af17a8a33a34897015469605623d19e23b2d167a5c67b"} Jul 12 17:54:03.202362 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.202321 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:03.202530 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.202428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:03.202608 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:03.202502 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:54:03.202608 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:03.202565 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:54:03.202608 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:03.202571 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b958f7c4c-wlvg6: secret "image-registry-tls" not found Jul 12 17:54:03.202750 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:03.202622 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls podName:cee3c823-d6f7-4498-b550-bd04f28d6152 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:05.202604652 +0000 UTC m=+38.846447376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls") pod "dns-default-nzstl" (UID: "cee3c823-d6f7-4498-b550-bd04f28d6152") : secret "dns-default-metrics-tls" not found Jul 12 17:54:03.202750 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:03.202638 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls podName:731149e1-0bb8-440e-a36f-a7f65052bf23 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:05.202631045 +0000 UTC m=+38.846473751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls") pod "image-registry-7b958f7c4c-wlvg6" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23") : secret "image-registry-tls" not found Jul 12 17:54:03.601622 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.601581 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:03.892455 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.892248 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94"] Jul 12 17:54:03.895502 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.895483 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" Jul 12 17:54:03.895776 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.895749 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:03.897804 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.897784 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zfgbg\"" Jul 12 17:54:03.897893 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.897845 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 12 17:54:03.898825 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.898812 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 12 17:54:03.902184 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.902150 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94"] Jul 12 17:54:03.906147 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.906122 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m4mq\" (UniqueName: \"kubernetes.io/projected/d6c13874-8138-412d-a1ca-ad2b1c9e6440-kube-api-access-4m4mq\") pod \"migrator-7f59b54f6c-q6v94\" (UID: \"d6c13874-8138-412d-a1ca-ad2b1c9e6440\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" Jul 12 17:54:03.960476 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:03.960425 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:04.006496 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:04.006460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m4mq\" (UniqueName: \"kubernetes.io/projected/d6c13874-8138-412d-a1ca-ad2b1c9e6440-kube-api-access-4m4mq\") pod \"migrator-7f59b54f6c-q6v94\" (UID: \"d6c13874-8138-412d-a1ca-ad2b1c9e6440\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" Jul 12 17:54:04.018335 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:04.018305 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m4mq\" (UniqueName: \"kubernetes.io/projected/d6c13874-8138-412d-a1ca-ad2b1c9e6440-kube-api-access-4m4mq\") pod \"migrator-7f59b54f6c-q6v94\" (UID: \"d6c13874-8138-412d-a1ca-ad2b1c9e6440\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" Jul 12 17:54:04.025830 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:04.025795 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:04.065062 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:04.065032 2568 generic.go:356] "Generic (PLEG): container finished" podID="950b9259-0b9e-4847-838d-26aad602908c" containerID="680186b16e062fa02b45c251ee2170d09611fa38347c58fee3ec0b7b051250f7" exitCode=0 Jul 12 17:54:04.065209 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:04.065099 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" event={"ID":"950b9259-0b9e-4847-838d-26aad602908c","Type":"ContainerDied","Data":"680186b16e062fa02b45c251ee2170d09611fa38347c58fee3ec0b7b051250f7"} Jul 12 17:54:04.205103 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:04.205073 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" Jul 12 17:54:04.881931 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:04.881776 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94"] Jul 12 17:54:04.887683 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:04.887653 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c13874_8138_412d_a1ca_ad2b1c9e6440.slice/crio-df16c4247a128eb7d7d2d22c539bf3341993c3d885d5e9e7d74c3b3e93ac4e96 WatchSource:0}: Error finding container df16c4247a128eb7d7d2d22c539bf3341993c3d885d5e9e7d74c3b3e93ac4e96: Status 404 returned error can't find the container with id df16c4247a128eb7d7d2d22c539bf3341993c3d885d5e9e7d74c3b3e93ac4e96 Jul 12 17:54:05.070960 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.070854 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" event={"ID":"950b9259-0b9e-4847-838d-26aad602908c","Type":"ContainerStarted","Data":"6a582164b1224fbcfa2b3d8e0a56859fae7983d90fc321cd01873177799fdd82"} Jul 12 17:54:05.071996 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.071977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" event={"ID":"d6c13874-8138-412d-a1ca-ad2b1c9e6440","Type":"ContainerStarted","Data":"df16c4247a128eb7d7d2d22c539bf3341993c3d885d5e9e7d74c3b3e93ac4e96"} Jul 12 17:54:05.073398 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.073371 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jvjpf" event={"ID":"68db3a53-54e6-4a52-9b0b-63f9aedb0c83","Type":"ContainerStarted","Data":"74d0726e1d72263b69744cff2263e0116ccc0a600859e16585e21cc6de206f2f"} Jul 12 17:54:05.098921 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.098874 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wxmk8" podStartSLOduration=8.060568797 podStartE2EDuration="38.098861138s" totalImagesPullingTime="30.038292341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:05.098670771 +0000 UTC m=+38.742513494" watchObservedRunningTime="2026-07-12 17:54:05.098861138 +0000 UTC m=+38.742703862" Jul 12 17:54:05.116866 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.116830 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jvjpf" podStartSLOduration=1.873671485 podStartE2EDuration="4.116816766s" totalImagesPullingTime="2.243145281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:05.115802881 +0000 UTC m=+38.759645607" watchObservedRunningTime="2026-07-12 17:54:05.116816766 +0000 UTC m=+38.760659502" Jul 12 17:54:05.217174 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.217143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:05.217323 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.217201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:05.217323 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:05.217296 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:54:05.217323 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:05.217300 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:54:05.217323 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:05.217320 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b958f7c4c-wlvg6: secret "image-registry-tls" not found Jul 12 17:54:05.217463 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:05.217350 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls podName:cee3c823-d6f7-4498-b550-bd04f28d6152 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:09.217334246 +0000 UTC m=+42.861176949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls") pod "dns-default-nzstl" (UID: "cee3c823-d6f7-4498-b550-bd04f28d6152") : secret "dns-default-metrics-tls" not found Jul 12 17:54:05.217463 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:05.217364 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls podName:731149e1-0bb8-440e-a36f-a7f65052bf23 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:09.21735815 +0000 UTC m=+42.861200850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls") pod "image-registry-7b958f7c4c-wlvg6" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23") : secret "image-registry-tls" not found Jul 12 17:54:05.232306 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.232273 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:05.335351 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.335267 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:05.432672 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.432647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:05.498244 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.498207 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:05.580015 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.579970 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:05.680671 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.680639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:05.858016 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.857988 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:05.960442 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:05.960361 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:06.062723 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:06.062692 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:07.079093 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.079060 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" event={"ID":"d6c13874-8138-412d-a1ca-ad2b1c9e6440","Type":"ContainerStarted","Data":"a5b5f20b9fe0168499c81ff9998ba3931fa646f462e485144b17a788f429a53d"} Jul 12 17:54:07.079469 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.079108 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" event={"ID":"d6c13874-8138-412d-a1ca-ad2b1c9e6440","Type":"ContainerStarted","Data":"4361bae381a235770e9cd2bc6ba0834a364e23fac804032222c207df47cfeb64"} Jul 12 17:54:07.094294 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.094250 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q6v94" podStartSLOduration=2.763461034 podStartE2EDuration="4.094235697s" totalImagesPullingTime="1.330774663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:07.093846637 +0000 UTC m=+40.737689361" watchObservedRunningTime="2026-07-12 17:54:07.094235697 +0000 UTC m=+40.738078420" Jul 12 17:54:07.172667 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.172638 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:07.227764 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.227740 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:07.476176 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.476143 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cxlbn"] Jul 12 17:54:07.482312 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.482284 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.484907 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.484877 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 12 17:54:07.485020 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.484960 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 12 17:54:07.485162 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.485143 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wcmql\"" Jul 12 17:54:07.485944 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.485931 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 12 17:54:07.485991 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.485930 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 12 17:54:07.495104 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.495082 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cxlbn"] Jul 12 17:54:07.526007 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.525978 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:07.536235 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.536217 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/869ca900-e02e-44ce-b839-f426fcdd3623-signing-cabundle\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.536318 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.536243 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqc5b\" (UniqueName: \"kubernetes.io/projected/869ca900-e02e-44ce-b839-f426fcdd3623-kube-api-access-zqc5b\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.536318 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.536280 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/869ca900-e02e-44ce-b839-f426fcdd3623-signing-key\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.596148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.596117 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:07.637534 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.637496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/869ca900-e02e-44ce-b839-f426fcdd3623-signing-cabundle\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.637637 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.637544 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqc5b\" (UniqueName: \"kubernetes.io/projected/869ca900-e02e-44ce-b839-f426fcdd3623-kube-api-access-zqc5b\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.637693 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.637680 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/869ca900-e02e-44ce-b839-f426fcdd3623-signing-key\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.638183 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.638162 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/869ca900-e02e-44ce-b839-f426fcdd3623-signing-cabundle\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.640855 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.640836 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/869ca900-e02e-44ce-b839-f426fcdd3623-signing-key\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.646103 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.646082 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqc5b\" (UniqueName: \"kubernetes.io/projected/869ca900-e02e-44ce-b839-f426fcdd3623-kube-api-access-zqc5b\") pod \"service-ca-79f68b9f4f-cxlbn\" (UID: \"869ca900-e02e-44ce-b839-f426fcdd3623\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.791246 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.791171 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" Jul 12 17:54:07.912414 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:07.912384 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cxlbn"] Jul 12 17:54:07.915713 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:07.915683 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869ca900_e02e_44ce_b839_f426fcdd3623.slice/crio-8a9e56201c74cbf6df5adb20a546559e87797fb9a6e070dee2805c77cc8a8fae WatchSource:0}: Error finding container 8a9e56201c74cbf6df5adb20a546559e87797fb9a6e070dee2805c77cc8a8fae: Status 404 returned error can't find the container with id 8a9e56201c74cbf6df5adb20a546559e87797fb9a6e070dee2805c77cc8a8fae Jul 12 17:54:08.082349 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:08.082274 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" event={"ID":"869ca900-e02e-44ce-b839-f426fcdd3623","Type":"ContainerStarted","Data":"8a9e56201c74cbf6df5adb20a546559e87797fb9a6e070dee2805c77cc8a8fae"} Jul 12 17:54:08.498715 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:08.498677 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:09.251408 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:09.251376 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:09.251758 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:09.251431 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:09.251758 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:09.251536 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 17:54:09.251758 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:09.251539 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 17:54:09.251758 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:09.251559 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7b958f7c4c-wlvg6: secret "image-registry-tls" not found Jul 12 17:54:09.251758 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:09.251587 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls podName:cee3c823-d6f7-4498-b550-bd04f28d6152 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:17.251573129 +0000 UTC m=+50.895415833 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls") pod "dns-default-nzstl" (UID: "cee3c823-d6f7-4498-b550-bd04f28d6152") : secret "dns-default-metrics-tls" not found Jul 12 17:54:09.251758 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:09.251600 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls podName:731149e1-0bb8-440e-a36f-a7f65052bf23 nodeName:}" failed. No retries permitted until 2026-07-12 17:54:17.251594423 +0000 UTC m=+50.895437124 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls") pod "image-registry-7b958f7c4c-wlvg6" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23") : secret "image-registry-tls" not found Jul 12 17:54:11.090685 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:11.090648 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" event={"ID":"869ca900-e02e-44ce-b839-f426fcdd3623","Type":"ContainerStarted","Data":"85bcfa2432de6f033948fccc24be08ac4a66664b1aae1f4ba54bb564e0a25c71"} Jul 12 17:54:11.108051 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:11.108006 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-cxlbn" podStartSLOduration=1.79982783 podStartE2EDuration="4.107991454s" totalImagesPullingTime="2.308163624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:11.107701291 +0000 UTC m=+44.751544014" watchObservedRunningTime="2026-07-12 17:54:11.107991454 +0000 UTC m=+44.751834177" Jul 12 17:54:11.427707 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:11.427674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:11.972074 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:11.972044 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:12.272022 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:12.271940 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:12.376864 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:12.376827 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:12.475582 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:12.475359 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:12.549521 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:12.549421 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:12.629698 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:12.629664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:12.865247 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:12.865138 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:13.064902 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:13.064867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:14.055400 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:14.055369 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:14.459462 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:14.459395 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:14.661826 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:14.661795 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:14.858553 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:14.858439 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:15.061192 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:15.061148 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:15.868579 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:15.868545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:15.967542 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:15.967502 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:16.263601 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:16.263569 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:16.465081 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:16.465049 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:16.664745 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:16.664716 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:16.721576 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:16.721545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:16.868856 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:16.868824 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:17.067922 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.067847 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:17.318485 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.318405 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:17.318485 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.318476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:17.320698 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.320674 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"image-registry-7b958f7c4c-wlvg6\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:17.320787 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.320711 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cee3c823-d6f7-4498-b550-bd04f28d6152-metrics-tls\") pod \"dns-default-nzstl\" (UID: \"cee3c823-d6f7-4498-b550-bd04f28d6152\") " pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:17.368308 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.368280 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:17.399773 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.399751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:17.499726 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.499700 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b958f7c4c-wlvg6"] Jul 12 17:54:17.503991 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:17.503962 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731149e1_0bb8_440e_a36f_a7f65052bf23.slice/crio-a9e943cb6ac13e78ff00bd724eb8923e18359761e84a9fb8ab5571260049cce1 WatchSource:0}: Error finding container a9e943cb6ac13e78ff00bd724eb8923e18359761e84a9fb8ab5571260049cce1: Status 404 returned error can't find the container with id a9e943cb6ac13e78ff00bd724eb8923e18359761e84a9fb8ab5571260049cce1 Jul 12 17:54:17.541391 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:17.540257 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nzstl"] Jul 12 17:54:17.553491 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:17.553461 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcee3c823_d6f7_4498_b550_bd04f28d6152.slice/crio-ab3eaeb926fd1c68c1e6e76a0075ab0131a12af6705e72c0096d2cbda2c9607b WatchSource:0}: Error finding container ab3eaeb926fd1c68c1e6e76a0075ab0131a12af6705e72c0096d2cbda2c9607b: Status 404 returned error can't find the container with id ab3eaeb926fd1c68c1e6e76a0075ab0131a12af6705e72c0096d2cbda2c9607b Jul 12 17:54:18.108563 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.108520 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzstl" event={"ID":"cee3c823-d6f7-4498-b550-bd04f28d6152","Type":"ContainerStarted","Data":"ab3eaeb926fd1c68c1e6e76a0075ab0131a12af6705e72c0096d2cbda2c9607b"} Jul 12 17:54:18.109799 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.109778 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" event={"ID":"731149e1-0bb8-440e-a36f-a7f65052bf23","Type":"ContainerStarted","Data":"57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7"} Jul 12 17:54:18.109890 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.109813 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" event={"ID":"731149e1-0bb8-440e-a36f-a7f65052bf23","Type":"ContainerStarted","Data":"a9e943cb6ac13e78ff00bd724eb8923e18359761e84a9fb8ab5571260049cce1"} Jul 12 17:54:18.109935 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.109910 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:18.129066 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.129019 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" podStartSLOduration=56.12900555 podStartE2EDuration="56.12900555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:18.128974061 +0000 UTC m=+51.772816786" watchObservedRunningTime="2026-07-12 17:54:18.12900555 +0000 UTC m=+51.772848263" Jul 12 17:54:18.258121 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.258082 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:18.358992 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.358903 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:18.462641 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.462613 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:18.665269 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.665236 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:18.860646 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:18.860610 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:19.058732 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:19.058647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:19.549442 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:19.549401 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:19.604290 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:19.604263 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:19.978874 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:19.978850 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:20.079895 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:20.079868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:20.119075 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:20.119030 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzstl" event={"ID":"cee3c823-d6f7-4498-b550-bd04f28d6152","Type":"ContainerStarted","Data":"c3078df76e2e9a7b09446f5a3827bd81edcb99b308f26122de9e3a69e63d5597"} Jul 12 17:54:20.119227 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:20.119089 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nzstl" event={"ID":"cee3c823-d6f7-4498-b550-bd04f28d6152","Type":"ContainerStarted","Data":"e9ebab6da059a67e24dcb33f2f31f946bc34c16f6bb7d493a8888420f6c5dafb"} Jul 12 17:54:20.119227 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:20.119172 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:20.137041 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:20.136994 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nzstl" podStartSLOduration=16.858518412 podStartE2EDuration="19.136978352s" totalImagesPullingTime="2.27845994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:20.135368648 +0000 UTC m=+53.779211371" watchObservedRunningTime="2026-07-12 17:54:20.136978352 +0000 UTC m=+53.780821120" Jul 12 17:54:20.267148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:20.267115 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:20.464295 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:20.464268 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:20.865718 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:20.865644 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:21.466707 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:21.466674 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:21.864829 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:21.864749 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:22.063897 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:22.063870 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:22.258314 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:22.258280 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:26.557738 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:26.557696 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:26.658029 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:26.657998 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:26.717095 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:26.717068 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:27.046871 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:27.046847 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4j476" Jul 12 17:54:27.069333 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:27.069303 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:27.240711 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:27.240677 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:27.448324 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:27.448293 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:28.052109 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:28.052079 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:28.197372 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:28.197343 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:28.298990 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:28.298960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:28.398059 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:28.398031 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:29.658145 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.658119 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-qtd2t"] Jul 12 17:54:29.660880 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.660863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.664900 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.664867 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 17:54:29.665087 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.665065 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 17:54:29.665525 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.665492 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 12 17:54:29.665525 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.665516 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 12 17:54:29.665667 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.665521 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-rl7pz\"" Jul 12 17:54:29.672755 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.669933 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 12 17:54:29.674063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.674044 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-qtd2t"] Jul 12 17:54:29.708161 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.708137 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74c2ade1-1a3b-4385-896c-daa000f2538b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.708282 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.708166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rznhr\" (UniqueName: \"kubernetes.io/projected/74c2ade1-1a3b-4385-896c-daa000f2538b-kube-api-access-rznhr\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.708282 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.708188 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74c2ade1-1a3b-4385-896c-daa000f2538b-serving-cert\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.708389 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.708286 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/74c2ade1-1a3b-4385-896c-daa000f2538b-snapshots\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.708389 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.708348 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74c2ade1-1a3b-4385-896c-daa000f2538b-tmp\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.708389 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.708365 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74c2ade1-1a3b-4385-896c-daa000f2538b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.767370 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.767340 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm"] Jul 12 17:54:29.769461 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.769446 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:29.772147 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.772124 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:29.772985 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.772964 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 12 17:54:29.773067 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.772984 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 17:54:29.773208 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.773194 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 17:54:29.776043 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.776026 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79486c5bd6-wvmlj"] Jul 12 17:54:29.778206 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.778187 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:29.778848 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.778831 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-m482m\"" Jul 12 17:54:29.780412 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.780393 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 12 17:54:29.783944 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.783927 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 12 17:54:29.784040 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.783945 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 12 17:54:29.784040 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.783955 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 17:54:29.784040 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.784016 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rn6tq\"" Jul 12 17:54:29.784197 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.784018 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 12 17:54:29.784197 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.784053 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 17:54:29.784283 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.784259 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 12 17:54:29.785437 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.785418 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-sns9z"] Jul 12 17:54:29.787768 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.787755 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-sns9z" Jul 12 17:54:29.793617 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.793596 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm"] Jul 12 17:54:29.798352 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.798335 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-v67rb\"" Jul 12 17:54:29.798437 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.798340 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 17:54:29.799135 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.799121 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 17:54:29.803412 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.803393 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-sns9z"] Jul 12 17:54:29.808797 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.808777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/74c2ade1-1a3b-4385-896c-daa000f2538b-snapshots\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.808891 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.808846 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74c2ade1-1a3b-4385-896c-daa000f2538b-tmp\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.808891 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.808876 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74c2ade1-1a3b-4385-896c-daa000f2538b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.809005 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.808917 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74c2ade1-1a3b-4385-896c-daa000f2538b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.809118 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.809094 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rznhr\" (UniqueName: \"kubernetes.io/projected/74c2ade1-1a3b-4385-896c-daa000f2538b-kube-api-access-rznhr\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.809191 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.809134 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74c2ade1-1a3b-4385-896c-daa000f2538b-serving-cert\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.809305 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.809290 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74c2ade1-1a3b-4385-896c-daa000f2538b-tmp\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.809405 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.809390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/74c2ade1-1a3b-4385-896c-daa000f2538b-snapshots\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.810108 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.810086 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74c2ade1-1a3b-4385-896c-daa000f2538b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.810405 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.810390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74c2ade1-1a3b-4385-896c-daa000f2538b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.811917 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.811900 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/74c2ade1-1a3b-4385-896c-daa000f2538b-serving-cert\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.832091 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.832063 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rznhr\" (UniqueName: \"kubernetes.io/projected/74c2ade1-1a3b-4385-896c-daa000f2538b-kube-api-access-rznhr\") pod \"insights-operator-68d9dd94cf-qtd2t\" (UID: \"74c2ade1-1a3b-4385-896c-daa000f2538b\") " pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.845624 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.845605 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79486c5bd6-wvmlj"] Jul 12 17:54:29.905704 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.905673 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:29.909782 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.909719 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxs8z\" (UniqueName: \"kubernetes.io/projected/2fa32b08-267f-4092-b626-c4b0908a4d42-kube-api-access-fxs8z\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:29.909782 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.909774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fa32b08-267f-4092-b626-c4b0908a4d42-service-ca-bundle\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:29.909956 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.909801 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58xw6\" (UniqueName: \"kubernetes.io/projected/749f64c4-70ba-42ac-aa63-ba470c87b5a5-kube-api-access-58xw6\") pod \"downloads-68c9cbb497-sns9z\" (UID: \"749f64c4-70ba-42ac-aa63-ba470c87b5a5\") " pod="openshift-console/downloads-68c9cbb497-sns9z" Jul 12 17:54:29.909956 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.909874 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lmgs\" (UniqueName: \"kubernetes.io/projected/521e82e9-be97-48e8-98cc-912f83b22c23-kube-api-access-6lmgs\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:29.909956 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.909919 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-default-certificate\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:29.909956 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.909949 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/521e82e9-be97-48e8-98cc-912f83b22c23-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:29.910150 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.909980 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-metrics-certs\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:29.910150 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.910007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-stats-auth\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:29.910150 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.910063 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/521e82e9-be97-48e8-98cc-912f83b22c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:29.970914 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.970883 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" Jul 12 17:54:29.982089 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:29.982056 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:30.010910 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.010870 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxs8z\" (UniqueName: \"kubernetes.io/projected/2fa32b08-267f-4092-b626-c4b0908a4d42-kube-api-access-fxs8z\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.011076 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.010928 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fa32b08-267f-4092-b626-c4b0908a4d42-service-ca-bundle\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.011076 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.010956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58xw6\" (UniqueName: \"kubernetes.io/projected/749f64c4-70ba-42ac-aa63-ba470c87b5a5-kube-api-access-58xw6\") pod \"downloads-68c9cbb497-sns9z\" (UID: \"749f64c4-70ba-42ac-aa63-ba470c87b5a5\") " pod="openshift-console/downloads-68c9cbb497-sns9z" Jul 12 17:54:30.011076 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.010980 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lmgs\" (UniqueName: \"kubernetes.io/projected/521e82e9-be97-48e8-98cc-912f83b22c23-kube-api-access-6lmgs\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:30.011076 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.011023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-default-certificate\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.011076 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.011053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/521e82e9-be97-48e8-98cc-912f83b22c23-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:30.011585 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.011082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-metrics-certs\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.011585 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.011108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-stats-auth\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.011585 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.011139 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/521e82e9-be97-48e8-98cc-912f83b22c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:30.011784 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.011761 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fa32b08-267f-4092-b626-c4b0908a4d42-service-ca-bundle\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.011959 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.011933 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/521e82e9-be97-48e8-98cc-912f83b22c23-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:30.014073 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.014045 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-default-certificate\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.014368 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.014349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/521e82e9-be97-48e8-98cc-912f83b22c23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:30.014443 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.014389 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-stats-auth\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.014788 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.014769 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fa32b08-267f-4092-b626-c4b0908a4d42-metrics-certs\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.023712 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.023033 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxs8z\" (UniqueName: \"kubernetes.io/projected/2fa32b08-267f-4092-b626-c4b0908a4d42-kube-api-access-fxs8z\") pod \"router-default-79486c5bd6-wvmlj\" (UID: \"2fa32b08-267f-4092-b626-c4b0908a4d42\") " pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.024614 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.024566 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lmgs\" (UniqueName: \"kubernetes.io/projected/521e82e9-be97-48e8-98cc-912f83b22c23-kube-api-access-6lmgs\") pod \"cluster-monitoring-operator-5f7f494dd4-zpnbm\" (UID: \"521e82e9-be97-48e8-98cc-912f83b22c23\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:30.025021 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.025001 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58xw6\" (UniqueName: \"kubernetes.io/projected/749f64c4-70ba-42ac-aa63-ba470c87b5a5-kube-api-access-58xw6\") pod \"downloads-68c9cbb497-sns9z\" (UID: \"749f64c4-70ba-42ac-aa63-ba470c87b5a5\") " pod="openshift-console/downloads-68c9cbb497-sns9z" Jul 12 17:54:30.079265 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.079240 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" Jul 12 17:54:30.082673 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.082649 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:30.087260 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.087242 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:30.095957 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.095889 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-qtd2t"] Jul 12 17:54:30.097625 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.097467 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-sns9z" Jul 12 17:54:30.098969 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:30.098949 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74c2ade1_1a3b_4385_896c_daa000f2538b.slice/crio-b7ff1d97d46a295532e4cf9ae6f9e7acb8199d9abe0b714535194a664926f480 WatchSource:0}: Error finding container b7ff1d97d46a295532e4cf9ae6f9e7acb8199d9abe0b714535194a664926f480: Status 404 returned error can't find the container with id b7ff1d97d46a295532e4cf9ae6f9e7acb8199d9abe0b714535194a664926f480 Jul 12 17:54:30.128144 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.128097 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nzstl" Jul 12 17:54:30.154815 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.154737 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" event={"ID":"74c2ade1-1a3b-4385-896c-daa000f2538b","Type":"ContainerStarted","Data":"b7ff1d97d46a295532e4cf9ae6f9e7acb8199d9abe0b714535194a664926f480"} Jul 12 17:54:30.223324 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.223298 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm"] Jul 12 17:54:30.225762 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:30.225725 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521e82e9_be97_48e8_98cc_912f83b22c23.slice/crio-65c218a1af1ad4aef9cae7bb591e9bef4d7c011ca6368522faacc8aa6e0efac2 WatchSource:0}: Error finding container 65c218a1af1ad4aef9cae7bb591e9bef4d7c011ca6368522faacc8aa6e0efac2: Status 404 returned error can't find the container with id 65c218a1af1ad4aef9cae7bb591e9bef4d7c011ca6368522faacc8aa6e0efac2 Jul 12 17:54:30.242868 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.242845 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79486c5bd6-wvmlj"] Jul 12 17:54:30.245931 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:30.245897 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fa32b08_267f_4092_b626_c4b0908a4d42.slice/crio-04ed1743d73982951a380f6036c2358147fdfbeb92cb9f51f0494961bf88a203 WatchSource:0}: Error finding container 04ed1743d73982951a380f6036c2358147fdfbeb92cb9f51f0494961bf88a203: Status 404 returned error can't find the container with id 04ed1743d73982951a380f6036c2358147fdfbeb92cb9f51f0494961bf88a203 Jul 12 17:54:30.270419 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:30.270391 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-sns9z"] Jul 12 17:54:30.280431 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:30.280406 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749f64c4_70ba_42ac_aa63_ba470c87b5a5.slice/crio-3999f450655f5bb726cf82205777c1c614107f05e34cf628d01ed65f49315f93 WatchSource:0}: Error finding container 3999f450655f5bb726cf82205777c1c614107f05e34cf628d01ed65f49315f93: Status 404 returned error can't find the container with id 3999f450655f5bb726cf82205777c1c614107f05e34cf628d01ed65f49315f93 Jul 12 17:54:31.161267 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:31.161229 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" event={"ID":"2fa32b08-267f-4092-b626-c4b0908a4d42","Type":"ContainerStarted","Data":"c63ded49603bea51205ca0d81a5bae52ce35aea9689acc8d08305e250aaea5e5"} Jul 12 17:54:31.161745 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:31.161294 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" event={"ID":"2fa32b08-267f-4092-b626-c4b0908a4d42","Type":"ContainerStarted","Data":"04ed1743d73982951a380f6036c2358147fdfbeb92cb9f51f0494961bf88a203"} Jul 12 17:54:31.166637 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:31.164966 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" event={"ID":"521e82e9-be97-48e8-98cc-912f83b22c23","Type":"ContainerStarted","Data":"65c218a1af1ad4aef9cae7bb591e9bef4d7c011ca6368522faacc8aa6e0efac2"} Jul 12 17:54:31.168161 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:31.168097 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-sns9z" event={"ID":"749f64c4-70ba-42ac-aa63-ba470c87b5a5","Type":"ContainerStarted","Data":"3999f450655f5bb726cf82205777c1c614107f05e34cf628d01ed65f49315f93"} Jul 12 17:54:31.187012 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:31.186825 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" podStartSLOduration=2.18680711 podStartE2EDuration="2.18680711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:31.18497806 +0000 UTC m=+64.828820784" watchObservedRunningTime="2026-07-12 17:54:31.18680711 +0000 UTC m=+64.830649834" Jul 12 17:54:32.088295 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.088261 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:32.091126 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.091103 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:32.173519 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.173477 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:32.174964 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.174901 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79486c5bd6-wvmlj" Jul 12 17:54:32.533878 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.533836 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:54:32.536546 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.536484 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 17:54:32.547170 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.547144 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d907cd33-4123-4333-a38a-ac8e60b5ef04-original-pull-secret\") pod \"global-pull-secret-syncer-djndg\" (UID: \"d907cd33-4123-4333-a38a-ac8e60b5ef04\") " pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:54:32.634738 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.634696 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:54:32.634945 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.634774 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:54:32.637887 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.637855 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 17:54:32.637887 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.637876 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 17:54:32.647392 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.647373 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 17:54:32.647523 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.647373 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ee4bc0-6991-412a-884c-dbc5e5ef7665-metrics-certs\") pod \"network-metrics-daemon-nxhsg\" (UID: \"d6ee4bc0-6991-412a-884c-dbc5e5ef7665\") " pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:54:32.657813 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.657791 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k29d\" (UniqueName: \"kubernetes.io/projected/fde9c63f-8a31-4bb1-9170-d280505e5b97-kube-api-access-7k29d\") pod \"network-check-target-cwgzb\" (UID: \"fde9c63f-8a31-4bb1-9170-d280505e5b97\") " pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:54:32.740680 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.740655 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-djndg" Jul 12 17:54:32.750179 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.750156 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-84rf9\"" Jul 12 17:54:32.756072 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.756048 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h9zfm\"" Jul 12 17:54:32.757941 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.757905 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nxhsg" Jul 12 17:54:32.764927 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.764908 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:54:32.926462 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.926373 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-djndg"] Jul 12 17:54:32.948471 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.948427 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cwgzb"] Jul 12 17:54:32.952218 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:32.952180 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde9c63f_8a31_4bb1_9170_d280505e5b97.slice/crio-dbfa274d3b1054bc1c55e7eec5bcf96b94b94dcda6417044fd0abea566a0ef6f WatchSource:0}: Error finding container dbfa274d3b1054bc1c55e7eec5bcf96b94b94dcda6417044fd0abea566a0ef6f: Status 404 returned error can't find the container with id dbfa274d3b1054bc1c55e7eec5bcf96b94b94dcda6417044fd0abea566a0ef6f Jul 12 17:54:32.984746 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:32.984719 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nxhsg"] Jul 12 17:54:32.987984 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:32.987950 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ee4bc0_6991_412a_884c_dbc5e5ef7665.slice/crio-33647e8d7b47bf25057a8057d8123d9b407e44e4db528b96140eb7b761f55bdc WatchSource:0}: Error finding container 33647e8d7b47bf25057a8057d8123d9b407e44e4db528b96140eb7b761f55bdc: Status 404 returned error can't find the container with id 33647e8d7b47bf25057a8057d8123d9b407e44e4db528b96140eb7b761f55bdc Jul 12 17:54:33.100289 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.100209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:33.178112 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.178077 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cwgzb" event={"ID":"fde9c63f-8a31-4bb1-9170-d280505e5b97","Type":"ContainerStarted","Data":"dbfa274d3b1054bc1c55e7eec5bcf96b94b94dcda6417044fd0abea566a0ef6f"} Jul 12 17:54:33.179617 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.179593 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" event={"ID":"74c2ade1-1a3b-4385-896c-daa000f2538b","Type":"ContainerStarted","Data":"2bc0f42d37759108765eabe6cebc443d38bc739079159684205b388aea50940d"} Jul 12 17:54:33.181425 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.181402 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" event={"ID":"521e82e9-be97-48e8-98cc-912f83b22c23","Type":"ContainerStarted","Data":"b5a6a69161678e9b149a481d122d20f54e9ac8af7a420df73ebf344aca839f32"} Jul 12 17:54:33.182784 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.182761 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nxhsg" event={"ID":"d6ee4bc0-6991-412a-884c-dbc5e5ef7665","Type":"ContainerStarted","Data":"33647e8d7b47bf25057a8057d8123d9b407e44e4db528b96140eb7b761f55bdc"} Jul 12 17:54:33.184215 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.184191 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-djndg" event={"ID":"d907cd33-4123-4333-a38a-ac8e60b5ef04","Type":"ContainerStarted","Data":"1324f7a9ff67faaead785f91a6ccc5d3fcd5209fd8dad52a2b5a4766ca3821f5"} Jul 12 17:54:33.197979 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.197932 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-qtd2t" podStartSLOduration=1.58647765 podStartE2EDuration="4.197915222s" totalImagesPullingTime="2.611437572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:33.197004141 +0000 UTC m=+66.840846866" watchObservedRunningTime="2026-07-12 17:54:33.197915222 +0000 UTC m=+66.841757947" Jul 12 17:54:33.202529 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.202448 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:33.215545 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.215466 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zpnbm" podStartSLOduration=1.73541321 podStartE2EDuration="4.215446077s" totalImagesPullingTime="2.480032867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:33.214263644 +0000 UTC m=+66.858106369" watchObservedRunningTime="2026-07-12 17:54:33.215446077 +0000 UTC m=+66.859288801" Jul 12 17:54:33.289775 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.289741 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:33.289950 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.289821 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5"] Jul 12 17:54:33.294428 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.294403 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" Jul 12 17:54:33.297274 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.297153 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 12 17:54:33.298672 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.297327 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-j5szx\"" Jul 12 17:54:33.299522 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.299481 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5"] Jul 12 17:54:33.446005 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.445921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gtlc5\" (UID: \"aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" Jul 12 17:54:33.450958 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.450929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:33.559531 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.547180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gtlc5\" (UID: \"aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" Jul 12 17:54:33.559531 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.551956 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gtlc5\" (UID: \"aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" Jul 12 17:54:33.610525 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.610476 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" Jul 12 17:54:33.794297 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.794263 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5"] Jul 12 17:54:33.800974 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:33.800944 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2ec6df_14a1_4f35_82ed_2fbd6709f8dd.slice/crio-b2f78031abd725c56df3717832890c02a2d48a127d82a1dd8aabb0ced04fa408 WatchSource:0}: Error finding container b2f78031abd725c56df3717832890c02a2d48a127d82a1dd8aabb0ced04fa408: Status 404 returned error can't find the container with id b2f78031abd725c56df3717832890c02a2d48a127d82a1dd8aabb0ced04fa408 Jul 12 17:54:33.887295 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.887262 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-697dfd6b84-rz7xj"] Jul 12 17:54:33.891905 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.891878 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:33.904454 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.904428 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4rmzj\"" Jul 12 17:54:33.904792 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.904769 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 12 17:54:33.904902 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.904832 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 12 17:54:33.904957 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.904779 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 12 17:54:33.906591 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.906572 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 12 17:54:33.906960 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.906943 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 12 17:54:33.909682 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.909483 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697dfd6b84-rz7xj"] Jul 12 17:54:33.952352 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.952003 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-console-config\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:33.952352 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.952059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-oauth-serving-cert\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:33.952352 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.952093 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-service-ca\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:33.952352 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.952118 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzf8c\" (UniqueName: \"kubernetes.io/projected/b8f86227-49ba-4672-864d-1ec158f150c8-kube-api-access-wzf8c\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:33.952352 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.952163 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-serving-cert\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:33.952352 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:33.952188 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-oauth-config\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.053614 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.053104 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-oauth-serving-cert\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.053614 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.053152 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-service-ca\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.053614 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.053178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzf8c\" (UniqueName: \"kubernetes.io/projected/b8f86227-49ba-4672-864d-1ec158f150c8-kube-api-access-wzf8c\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.053614 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.053237 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-serving-cert\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.053614 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.053263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-oauth-config\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.053614 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.053317 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-console-config\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.055988 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.054808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-console-config\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.055988 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.054808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-oauth-serving-cert\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.055988 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.055684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-service-ca\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.058532 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.058475 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-serving-cert\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.059034 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.058997 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-oauth-config\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.063321 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.063296 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzf8c\" (UniqueName: \"kubernetes.io/projected/b8f86227-49ba-4672-864d-1ec158f150c8-kube-api-access-wzf8c\") pod \"console-697dfd6b84-rz7xj\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.191307 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.191245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" event={"ID":"aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd","Type":"ContainerStarted","Data":"b2f78031abd725c56df3717832890c02a2d48a127d82a1dd8aabb0ced04fa408"} Jul 12 17:54:34.211185 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.210581 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:34.638925 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.636629 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bjl5k"] Jul 12 17:54:34.642912 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.641429 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.648972 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.646408 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 17:54:34.651222 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.649274 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q8r2n\"" Jul 12 17:54:34.651222 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.649400 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 17:54:34.661422 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.661082 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bjl5k"] Jul 12 17:54:34.709223 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.709152 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697dfd6b84-rz7xj"] Jul 12 17:54:34.712131 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.712070 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:34.716527 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:34.716486 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f86227_49ba_4672_864d_1ec158f150c8.slice/crio-dd9a53ab1f0f14d061910834b3e12aa8621f9965dc2be3b8052ed3a481a0021f WatchSource:0}: Error finding container dd9a53ab1f0f14d061910834b3e12aa8621f9965dc2be3b8052ed3a481a0021f: Status 404 returned error can't find the container with id dd9a53ab1f0f14d061910834b3e12aa8621f9965dc2be3b8052ed3a481a0021f Jul 12 17:54:34.761970 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.761945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-crio-socket\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.762075 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.761987 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.762075 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.762018 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.762075 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.762047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsp82\" (UniqueName: \"kubernetes.io/projected/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-api-access-jsp82\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.762230 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.762124 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-data-volume\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.829326 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.828951 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:34.862822 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.862776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-crio-socket\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.862962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.862824 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.862962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.862865 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.862962 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.862896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsp82\" (UniqueName: \"kubernetes.io/projected/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-api-access-jsp82\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.863183 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.863006 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-data-volume\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.863840 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:34.863318 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:34.863840 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.863353 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-data-volume\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.863840 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:34.863398 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:35.36337641 +0000 UTC m=+69.007219126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : configmap "kube-rbac-proxy" not found Jul 12 17:54:34.863840 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.863456 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-crio-socket\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:34.863840 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:34.863471 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:34.863840 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:34.863537 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:35.363521788 +0000 UTC m=+69.007364504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:34.874059 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:34.872952 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsp82\" (UniqueName: \"kubernetes.io/projected/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-api-access-jsp82\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:35.201864 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.201325 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nxhsg" event={"ID":"d6ee4bc0-6991-412a-884c-dbc5e5ef7665","Type":"ContainerStarted","Data":"6dbf087f0a7dd47b3868c4d9c9087de81c5285ed975293a3c216c0d04baa3994"} Jul 12 17:54:35.201864 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.201392 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nxhsg" event={"ID":"d6ee4bc0-6991-412a-884c-dbc5e5ef7665","Type":"ContainerStarted","Data":"e43422fb0a24168ece2bc2118f53060ac798aa68832d512acc0f45ac7a34c0d2"} Jul 12 17:54:35.205570 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.205485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697dfd6b84-rz7xj" event={"ID":"b8f86227-49ba-4672-864d-1ec158f150c8","Type":"ContainerStarted","Data":"dd9a53ab1f0f14d061910834b3e12aa8621f9965dc2be3b8052ed3a481a0021f"} Jul 12 17:54:35.220172 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.219841 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nxhsg" podStartSLOduration=66.656110885 podStartE2EDuration="1m8.219807082s" totalImagesPullingTime="1.563696197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:35.21823314 +0000 UTC m=+68.862075876" watchObservedRunningTime="2026-07-12 17:54:35.219807082 +0000 UTC m=+68.863649800" Jul 12 17:54:35.237853 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.237783 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzstl_cee3c823-d6f7-4498-b550-bd04f28d6152/dns/0.log" Jul 12 17:54:35.335180 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.335118 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:35.368804 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.368582 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:35.368804 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.368664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:35.368804 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:35.368807 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:35.369090 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:35.368892 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:36.368871807 +0000 UTC m=+70.012714507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:35.369488 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:35.369347 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:35.369488 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:35.369434 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:36.369386947 +0000 UTC m=+70.013229649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : configmap "kube-rbac-proxy" not found Jul 12 17:54:35.435844 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.435799 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:35.437268 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.437147 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzstl_cee3c823-d6f7-4498-b550-bd04f28d6152/kube-rbac-proxy/0.log" Jul 12 17:54:35.773855 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.773823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:35.830817 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:35.830784 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:36.237956 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:36.237858 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d7cld_bcaa496d-87e7-4d2f-a1de-2a67abf5cb68/dns-node-resolver/0.log" Jul 12 17:54:36.378864 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:36.378619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:36.378864 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:36.378666 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:36.378864 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:36.378812 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:36.378864 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:36.378832 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:36.379190 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:36.378891 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:38.378870734 +0000 UTC m=+72.022713439 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : configmap "kube-rbac-proxy" not found Jul 12 17:54:36.379190 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:36.378916 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:38.378904799 +0000 UTC m=+72.022747505 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:36.837973 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:36.837912 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b958f7c4c-wlvg6_731149e1-0bb8-440e-a36f-a7f65052bf23/registry/0.log" Jul 12 17:54:37.218784 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:37.218746 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" event={"ID":"aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd","Type":"ContainerStarted","Data":"c8f5cf57c1259f04f5ae75dfc925f9b59ac3b84f819c48d3606e1af406c1b399"} Jul 12 17:54:37.219104 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:37.219084 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" Jul 12 17:54:37.225138 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:37.225115 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" Jul 12 17:54:37.235966 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:37.235899 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtlc5" podStartSLOduration=1.968060354 podStartE2EDuration="4.235881261s" totalImagesPullingTime="2.267820907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:37.234808018 +0000 UTC m=+70.878650742" watchObservedRunningTime="2026-07-12 17:54:37.235881261 +0000 UTC m=+70.879723965" Jul 12 17:54:37.238491 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:37.238441 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-498ks_d1dec3aa-5c23-4a97-8827-90a57806b2b2/node-ca/0.log" Jul 12 17:54:37.374400 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:37.374319 2568 patch_prober.go:28] interesting pod/image-registry-7b958f7c4c-wlvg6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 12 17:54:37.374582 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:37.374400 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" podUID="731149e1-0bb8-440e-a36f-a7f65052bf23" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 17:54:37.838466 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:37.838442 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79486c5bd6-wvmlj_2fa32b08-267f-4092-b626-c4b0908a4d42/router/0.log" Jul 12 17:54:38.238671 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:38.238643 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jvjpf_68db3a53-54e6-4a52-9b0b-63f9aedb0c83/serve-healthcheck-canary/0.log" Jul 12 17:54:38.303023 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:38.302967 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:38.362870 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:38.362836 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:38.394108 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:38.394043 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:38.394108 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:38.394090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:38.394345 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:38.394223 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:38.394345 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:38.394278 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:42.394261688 +0000 UTC m=+76.038104403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:38.394546 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:38.394500 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:38.394675 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:38.394585 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:42.394570339 +0000 UTC m=+76.038413040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : configmap "kube-rbac-proxy" not found Jul 12 17:54:38.715143 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:38.715110 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:39.118633 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:39.118561 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:54:39.252405 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:39.252350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:39.356203 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:39.356169 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:40.670041 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:40.670011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:40.768490 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:40.768430 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:40.869684 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:40.869652 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:41.237009 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.236960 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-djndg" event={"ID":"d907cd33-4123-4333-a38a-ac8e60b5ef04","Type":"ContainerStarted","Data":"85e9c1d6fea88a987dc6b7e4769c541f117ad8c49a0e227edc3d5607c473917a"} Jul 12 17:54:41.238480 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.238458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697dfd6b84-rz7xj" event={"ID":"b8f86227-49ba-4672-864d-1ec158f150c8","Type":"ContainerStarted","Data":"adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc"} Jul 12 17:54:41.239820 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.239794 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cwgzb" event={"ID":"fde9c63f-8a31-4bb1-9170-d280505e5b97","Type":"ContainerStarted","Data":"4b64245169a0d7457ea0943da4debb318ed2287dafc881e3477287474a4283e2"} Jul 12 17:54:41.239944 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.239928 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:54:41.252792 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.252744 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-djndg" podStartSLOduration=66.544181175 podStartE2EDuration="1m14.252724378s" totalImagesPullingTime="7.708543203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:41.252094639 +0000 UTC m=+74.895937363" watchObservedRunningTime="2026-07-12 17:54:41.252724378 +0000 UTC m=+74.896567096" Jul 12 17:54:41.268719 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.268654 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cwgzb" podStartSLOduration=66.582013195 podStartE2EDuration="1m14.268638902s" totalImagesPullingTime="7.686625707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:41.267963543 +0000 UTC m=+74.911806267" watchObservedRunningTime="2026-07-12 17:54:41.268638902 +0000 UTC m=+74.912481625" Jul 12 17:54:41.284215 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.284160 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-697dfd6b84-rz7xj" podStartSLOduration=2.354349733 podStartE2EDuration="8.284142868s" totalImagesPullingTime="5.929793135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:41.283276159 +0000 UTC m=+74.927118882" watchObservedRunningTime="2026-07-12 17:54:41.284142868 +0000 UTC m=+74.927985591" Jul 12 17:54:41.887728 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.887687 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:41.988961 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:41.988923 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:42.426647 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.426613 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:42.426647 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.426652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:42.426908 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:42.426744 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 17:54:42.426908 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:42.426761 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 17:54:42.426908 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:42.426812 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:50.42679492 +0000 UTC m=+84.070637625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : configmap "kube-rbac-proxy" not found Jul 12 17:54:42.426908 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:54:42.426825 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls podName:f8148f4a-8ab9-42c9-822e-8ba38b9274ed nodeName:}" failed. No retries permitted until 2026-07-12 17:54:50.426819445 +0000 UTC m=+84.070662145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bjl5k" (UID: "f8148f4a-8ab9-42c9-822e-8ba38b9274ed") : secret "insights-runtime-extractor-tls" not found Jul 12 17:54:42.447391 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.447359 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:42.529950 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.529914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:42.593047 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.593012 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:42.661802 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.661773 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:42.707407 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.707333 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-68wgp"] Jul 12 17:54:42.712081 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.712053 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.714838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.714722 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 17:54:42.714838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.714750 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 17:54:42.715246 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.715230 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wl5tq\"" Jul 12 17:54:42.715470 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.715452 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 17:54:42.715884 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.715862 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 17:54:42.830062 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-root\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.830244 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830072 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-tls\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.830244 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830105 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bfef6e6-74f1-41e6-acb2-fb369c86b753-metrics-client-ca\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.830244 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830133 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsxjd\" (UniqueName: \"kubernetes.io/projected/2bfef6e6-74f1-41e6-acb2-fb369c86b753-kube-api-access-jsxjd\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.830244 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-wtmp\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.830448 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830264 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-accelerators-collector-config\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.830448 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830294 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.830448 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-sys\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.830448 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.830378 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-textfile\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.931692 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-accelerators-collector-config\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931706 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-sys\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931779 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-textfile\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931818 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-root\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-tls\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931875 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bfef6e6-74f1-41e6-acb2-fb369c86b753-metrics-client-ca\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsxjd\" (UniqueName: \"kubernetes.io/projected/2bfef6e6-74f1-41e6-acb2-fb369c86b753-kube-api-access-jsxjd\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.931972 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-wtmp\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.932136 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-wtmp\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932626 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.932314 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-accelerators-collector-config\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932626 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.932377 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-root\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932626 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.932419 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2bfef6e6-74f1-41e6-acb2-fb369c86b753-sys\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932773 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.932686 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-textfile\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.932990 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.932960 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bfef6e6-74f1-41e6-acb2-fb369c86b753-metrics-client-ca\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.935585 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.935461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:42.935825 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.935660 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.936084 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.936063 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2bfef6e6-74f1-41e6-acb2-fb369c86b753-node-exporter-tls\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:42.944103 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:42.944077 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsxjd\" (UniqueName: \"kubernetes.io/projected/2bfef6e6-74f1-41e6-acb2-fb369c86b753-kube-api-access-jsxjd\") pod \"node-exporter-68wgp\" (UID: \"2bfef6e6-74f1-41e6-acb2-fb369c86b753\") " pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:43.025188 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:43.025110 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-68wgp" Jul 12 17:54:43.035839 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:43.035818 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:43.174222 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:43.174184 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:43.657760 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:43.657725 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:43.738876 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:43.738831 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:43.877692 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:43.877651 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:43.978268 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:43.978190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:44.211393 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:44.211362 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:44.211601 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:44.211460 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:44.216977 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:44.216948 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:44.256548 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:44.256444 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:54:45.590895 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:45.590867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:45.668846 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:45.668809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:45.878082 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:45.877938 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:45.978946 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:45.978911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:46.077971 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:46.077939 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:46.823938 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:46.823895 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:46.877839 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:46.877804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:47.458666 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:47.458627 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:47.559498 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:47.559461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:47.769761 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:47.769647 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:48.122004 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:48.121837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:48.222441 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:48.222404 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:48.275755 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:48.275717 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:48.866567 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:48.866539 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:49.943153 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:49.943123 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfef6e6_74f1_41e6_acb2_fb369c86b753.slice/crio-8eb979cadd331ea8001fd85472215cf55672692d63562e2ac0efbed0393925a0 WatchSource:0}: Error finding container 8eb979cadd331ea8001fd85472215cf55672692d63562e2ac0efbed0393925a0: Status 404 returned error can't find the container with id 8eb979cadd331ea8001fd85472215cf55672692d63562e2ac0efbed0393925a0 Jul 12 17:54:50.038947 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.038908 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:50.238995 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.238922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:50.274659 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.274632 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68wgp" event={"ID":"2bfef6e6-74f1-41e6-acb2-fb369c86b753","Type":"ContainerStarted","Data":"8eb979cadd331ea8001fd85472215cf55672692d63562e2ac0efbed0393925a0"} Jul 12 17:54:50.476858 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.476824 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:50.495916 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.495844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:50.495916 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.495893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:50.496446 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.496423 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:50.498524 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.498487 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f8148f4a-8ab9-42c9-822e-8ba38b9274ed-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bjl5k\" (UID: \"f8148f4a-8ab9-42c9-822e-8ba38b9274ed\") " pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:50.559889 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.559857 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bjl5k" Jul 12 17:54:50.959356 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:50.959323 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bjl5k"] Jul 12 17:54:51.166963 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:54:51.166776 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8148f4a_8ab9_42c9_822e_8ba38b9274ed.slice/crio-17ecbd60102d50a23542046d0d47a4ae409efb8b7143b9285b22f88ced1e7276 WatchSource:0}: Error finding container 17ecbd60102d50a23542046d0d47a4ae409efb8b7143b9285b22f88ced1e7276: Status 404 returned error can't find the container with id 17ecbd60102d50a23542046d0d47a4ae409efb8b7143b9285b22f88ced1e7276 Jul 12 17:54:51.281567 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:51.281535 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-sns9z" event={"ID":"749f64c4-70ba-42ac-aa63-ba470c87b5a5","Type":"ContainerStarted","Data":"284f5b8c87035a22bf6cbcc791d8e08fd36b7f3cf8d2289cb45e6b528717b3cc"} Jul 12 17:54:51.282986 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:51.282951 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bjl5k" event={"ID":"f8148f4a-8ab9-42c9-822e-8ba38b9274ed","Type":"ContainerStarted","Data":"17ecbd60102d50a23542046d0d47a4ae409efb8b7143b9285b22f88ced1e7276"} Jul 12 17:54:51.299790 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:51.299743 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-sns9z" podStartSLOduration=2.252042751 podStartE2EDuration="22.299725285s" totalImagesPullingTime="20.047682534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:54:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:51.298552841 +0000 UTC m=+84.942395568" watchObservedRunningTime="2026-07-12 17:54:51.299725285 +0000 UTC m=+84.943568007" Jul 12 17:54:51.689045 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:51.689011 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b958f7c4c-wlvg6"] Jul 12 17:54:51.782154 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:51.782077 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:51.881621 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:51.881564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:51.988936 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:51.988897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:52.301110 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:52.300397 2568 generic.go:356] "Generic (PLEG): container finished" podID="2bfef6e6-74f1-41e6-acb2-fb369c86b753" containerID="57f10a0a6acf10ae3a56be0f7cd00eda67c370a098b25698aafd1800f833261c" exitCode=0 Jul 12 17:54:52.301299 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:52.301219 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68wgp" event={"ID":"2bfef6e6-74f1-41e6-acb2-fb369c86b753","Type":"ContainerDied","Data":"57f10a0a6acf10ae3a56be0f7cd00eda67c370a098b25698aafd1800f833261c"} Jul 12 17:54:52.301873 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:52.301851 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-sns9z" Jul 12 17:54:52.314296 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:52.314277 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-sns9z" Jul 12 17:54:53.001571 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:53.001535 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:53.102584 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:53.102466 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:53.203236 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:53.203192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:53.309010 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:53.308905 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68wgp" event={"ID":"2bfef6e6-74f1-41e6-acb2-fb369c86b753","Type":"ContainerStarted","Data":"d38e8a1c656f3cb1831df32b016791f1ad49525ae800d6bee61e75c2549d0e8c"} Jul 12 17:54:53.309010 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:53.308968 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-68wgp" event={"ID":"2bfef6e6-74f1-41e6-acb2-fb369c86b753","Type":"ContainerStarted","Data":"6de393c318cf4b04c083fbab87575226d6cacda1e5d5b90a817b0bfd5018ab62"} Jul 12 17:54:53.329298 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:53.329193 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-68wgp" podStartSLOduration=9.981831159 podStartE2EDuration="11.329173314s" totalImagesPullingTime="1.249399474s" totalInitContainerRuntime="97.942681ms" isStatefulPod=true podCreationTimestamp="2026-07-12 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:54:53.32842633 +0000 UTC m=+86.972269055" watchObservedRunningTime="2026-07-12 17:54:53.329173314 +0000 UTC m=+86.973016046" Jul 12 17:54:54.086543 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:54.086445 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:54.187769 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:54.187733 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:57.198588 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:57.198558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:57.254663 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:57.254635 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:57.327777 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:57.327743 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bjl5k" event={"ID":"f8148f4a-8ab9-42c9-822e-8ba38b9274ed","Type":"ContainerStarted","Data":"9eedc1ec4afa255ec0b26a3b64f40c58efc02481822a68b1047c57a171480e11"} Jul 12 17:54:57.973561 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:57.973524 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697dfd6b84-rz7xj"] Jul 12 17:54:58.069849 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:58.069820 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:58.123426 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:58.123386 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:58.175736 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:58.175702 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:58.918999 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:58.918965 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:59.022840 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:59.022801 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:54:59.338470 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:54:59.338363 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bjl5k" event={"ID":"f8148f4a-8ab9-42c9-822e-8ba38b9274ed","Type":"ContainerStarted","Data":"b9bd2bf950dde3f7ca8679e216a8ca43657f5d8cc83d33b46e26ea26581cba08"} Jul 12 17:55:02.113539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:02.113449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:02.179786 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:02.179759 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:02.350684 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:02.350650 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bjl5k" event={"ID":"f8148f4a-8ab9-42c9-822e-8ba38b9274ed","Type":"ContainerStarted","Data":"1a92b7dc92cf9b0d06df7f443ba432b104d1fc6c6d11b74c306ef0194a34e4e9"} Jul 12 17:55:02.379819 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:02.377828 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bjl5k" podStartSLOduration=18.113965572 podStartE2EDuration="28.377808697s" totalImagesPullingTime="10.263843125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 17:54:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:55:02.375868155 +0000 UTC m=+96.019710881" watchObservedRunningTime="2026-07-12 17:55:02.377808697 +0000 UTC m=+96.021651421" Jul 12 17:55:04.546226 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:04.546192 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:04.646938 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:04.646909 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:12.247012 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:12.246982 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cwgzb" Jul 12 17:55:16.714861 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:16.714827 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" podUID="731149e1-0bb8-440e-a36f-a7f65052bf23" containerName="registry" containerID="cri-o://57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7" gracePeriod=30 Jul 12 17:55:16.972429 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:16.972372 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:55:17.013258 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013228 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-trusted-ca\") pod \"731149e1-0bb8-440e-a36f-a7f65052bf23\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " Jul 12 17:55:17.013392 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013265 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgdbq\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-kube-api-access-jgdbq\") pod \"731149e1-0bb8-440e-a36f-a7f65052bf23\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " Jul 12 17:55:17.013392 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013296 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-installation-pull-secrets\") pod \"731149e1-0bb8-440e-a36f-a7f65052bf23\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " Jul 12 17:55:17.013392 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013319 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-certificates\") pod \"731149e1-0bb8-440e-a36f-a7f65052bf23\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " Jul 12 17:55:17.013392 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013366 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") pod \"731149e1-0bb8-440e-a36f-a7f65052bf23\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " Jul 12 17:55:17.013628 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013407 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-image-registry-private-configuration\") pod \"731149e1-0bb8-440e-a36f-a7f65052bf23\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " Jul 12 17:55:17.013628 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013437 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-bound-sa-token\") pod \"731149e1-0bb8-440e-a36f-a7f65052bf23\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " Jul 12 17:55:17.013628 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013466 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/731149e1-0bb8-440e-a36f-a7f65052bf23-ca-trust-extracted\") pod \"731149e1-0bb8-440e-a36f-a7f65052bf23\" (UID: \"731149e1-0bb8-440e-a36f-a7f65052bf23\") " Jul 12 17:55:17.013778 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013680 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "731149e1-0bb8-440e-a36f-a7f65052bf23" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:55:17.013870 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.013836 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "731149e1-0bb8-440e-a36f-a7f65052bf23" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:55:17.015752 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.015726 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "731149e1-0bb8-440e-a36f-a7f65052bf23" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:55:17.015919 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.015889 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "731149e1-0bb8-440e-a36f-a7f65052bf23" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:55:17.016005 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.015917 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-kube-api-access-jgdbq" (OuterVolumeSpecName: "kube-api-access-jgdbq") pod "731149e1-0bb8-440e-a36f-a7f65052bf23" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23"). InnerVolumeSpecName "kube-api-access-jgdbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:55:17.016005 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.015986 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "731149e1-0bb8-440e-a36f-a7f65052bf23" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:55:17.016087 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.016014 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "731149e1-0bb8-440e-a36f-a7f65052bf23" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:55:17.022834 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.022811 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/731149e1-0bb8-440e-a36f-a7f65052bf23-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "731149e1-0bb8-440e-a36f-a7f65052bf23" (UID: "731149e1-0bb8-440e-a36f-a7f65052bf23"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:55:17.114455 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.114424 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-image-registry-private-configuration\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:17.114455 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.114451 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-bound-sa-token\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:17.114664 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.114467 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/731149e1-0bb8-440e-a36f-a7f65052bf23-ca-trust-extracted\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:17.114664 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.114478 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-trusted-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:17.114664 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.114489 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgdbq\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-kube-api-access-jgdbq\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:17.114664 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.114503 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/731149e1-0bb8-440e-a36f-a7f65052bf23-installation-pull-secrets\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:17.114664 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.114538 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-certificates\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:17.114664 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.114548 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/731149e1-0bb8-440e-a36f-a7f65052bf23-registry-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:17.401681 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.401649 2568 generic.go:356] "Generic (PLEG): container finished" podID="731149e1-0bb8-440e-a36f-a7f65052bf23" containerID="57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7" exitCode=0 Jul 12 17:55:17.401859 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.401692 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" Jul 12 17:55:17.401859 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.401699 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" event={"ID":"731149e1-0bb8-440e-a36f-a7f65052bf23","Type":"ContainerDied","Data":"57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7"} Jul 12 17:55:17.401859 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.401739 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b958f7c4c-wlvg6" event={"ID":"731149e1-0bb8-440e-a36f-a7f65052bf23","Type":"ContainerDied","Data":"a9e943cb6ac13e78ff00bd724eb8923e18359761e84a9fb8ab5571260049cce1"} Jul 12 17:55:17.401859 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.401757 2568 scope.go:117] "RemoveContainer" containerID="57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7" Jul 12 17:55:17.410580 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.410561 2568 scope.go:117] "RemoveContainer" containerID="57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7" Jul 12 17:55:17.410852 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:55:17.410832 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7\": container with ID starting with 57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7 not found: ID does not exist" containerID="57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7" Jul 12 17:55:17.410905 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.410863 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7"} err="failed to get container status \"57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7\": rpc error: code = NotFound desc = could not find container \"57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7\": container with ID starting with 57fda2bb8c6c39269f78e85c3e560a8c02c2e49049cfe02ad21a405a84d14ed7 not found: ID does not exist" Jul 12 17:55:17.422439 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.422416 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7b958f7c4c-wlvg6"] Jul 12 17:55:17.426397 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:17.426377 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7b958f7c4c-wlvg6"] Jul 12 17:55:18.833748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:18.833714 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="731149e1-0bb8-440e-a36f-a7f65052bf23" path="/var/lib/kubelet/pods/731149e1-0bb8-440e-a36f-a7f65052bf23/volumes" Jul 12 17:55:19.552147 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:19.552101 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:19.606275 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:19.606219 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:22.999758 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:22.999698 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-697dfd6b84-rz7xj" podUID="b8f86227-49ba-4672-864d-1ec158f150c8" containerName="console" containerID="cri-o://adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc" gracePeriod=15 Jul 12 17:55:23.280363 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.280340 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:55:23.361163 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361134 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-service-ca\") pod \"b8f86227-49ba-4672-864d-1ec158f150c8\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " Jul 12 17:55:23.361316 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361169 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-serving-cert\") pod \"b8f86227-49ba-4672-864d-1ec158f150c8\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " Jul 12 17:55:23.361316 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361201 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzf8c\" (UniqueName: \"kubernetes.io/projected/b8f86227-49ba-4672-864d-1ec158f150c8-kube-api-access-wzf8c\") pod \"b8f86227-49ba-4672-864d-1ec158f150c8\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " Jul 12 17:55:23.361316 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361224 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-oauth-config\") pod \"b8f86227-49ba-4672-864d-1ec158f150c8\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " Jul 12 17:55:23.361460 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361415 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-console-config\") pod \"b8f86227-49ba-4672-864d-1ec158f150c8\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " Jul 12 17:55:23.361539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361476 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-oauth-serving-cert\") pod \"b8f86227-49ba-4672-864d-1ec158f150c8\" (UID: \"b8f86227-49ba-4672-864d-1ec158f150c8\") " Jul 12 17:55:23.361690 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361653 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-service-ca" (OuterVolumeSpecName: "service-ca") pod "b8f86227-49ba-4672-864d-1ec158f150c8" (UID: "b8f86227-49ba-4672-864d-1ec158f150c8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:55:23.361810 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361762 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:23.361810 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361777 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-console-config" (OuterVolumeSpecName: "console-config") pod "b8f86227-49ba-4672-864d-1ec158f150c8" (UID: "b8f86227-49ba-4672-864d-1ec158f150c8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:55:23.361810 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.361781 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b8f86227-49ba-4672-864d-1ec158f150c8" (UID: "b8f86227-49ba-4672-864d-1ec158f150c8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:55:23.363629 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.363609 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b8f86227-49ba-4672-864d-1ec158f150c8" (UID: "b8f86227-49ba-4672-864d-1ec158f150c8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:55:23.363814 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.363796 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b8f86227-49ba-4672-864d-1ec158f150c8" (UID: "b8f86227-49ba-4672-864d-1ec158f150c8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:55:23.363854 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.363817 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8f86227-49ba-4672-864d-1ec158f150c8-kube-api-access-wzf8c" (OuterVolumeSpecName: "kube-api-access-wzf8c") pod "b8f86227-49ba-4672-864d-1ec158f150c8" (UID: "b8f86227-49ba-4672-864d-1ec158f150c8"). InnerVolumeSpecName "kube-api-access-wzf8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:55:23.422792 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.422766 2568 generic.go:356] "Generic (PLEG): container finished" podID="b8f86227-49ba-4672-864d-1ec158f150c8" containerID="adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc" exitCode=0 Jul 12 17:55:23.422905 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.422813 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697dfd6b84-rz7xj" Jul 12 17:55:23.422905 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.422847 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697dfd6b84-rz7xj" event={"ID":"b8f86227-49ba-4672-864d-1ec158f150c8","Type":"ContainerDied","Data":"adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc"} Jul 12 17:55:23.422905 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.422886 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697dfd6b84-rz7xj" event={"ID":"b8f86227-49ba-4672-864d-1ec158f150c8","Type":"ContainerDied","Data":"dd9a53ab1f0f14d061910834b3e12aa8621f9965dc2be3b8052ed3a481a0021f"} Jul 12 17:55:23.422905 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.422904 2568 scope.go:117] "RemoveContainer" containerID="adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc" Jul 12 17:55:23.430882 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.430865 2568 scope.go:117] "RemoveContainer" containerID="adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc" Jul 12 17:55:23.431135 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:55:23.431118 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc\": container with ID starting with adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc not found: ID does not exist" containerID="adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc" Jul 12 17:55:23.431182 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.431145 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc"} err="failed to get container status \"adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc\": rpc error: code = NotFound desc = could not find container \"adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc\": container with ID starting with adbc4161c440ce310476ba22985abb4fb63498071e2594c3b71c3cf6b8e6fdfc not found: ID does not exist" Jul 12 17:55:23.441778 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.441759 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697dfd6b84-rz7xj"] Jul 12 17:55:23.445732 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.445715 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-697dfd6b84-rz7xj"] Jul 12 17:55:23.463011 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.462991 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-console-config\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:23.463011 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.463010 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8f86227-49ba-4672-864d-1ec158f150c8-oauth-serving-cert\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:23.463141 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.463020 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-serving-cert\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:23.463141 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.463029 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzf8c\" (UniqueName: \"kubernetes.io/projected/b8f86227-49ba-4672-864d-1ec158f150c8-kube-api-access-wzf8c\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:23.463141 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:23.463037 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8f86227-49ba-4672-864d-1ec158f150c8-console-oauth-config\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:55:24.833934 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:24.833904 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8f86227-49ba-4672-864d-1ec158f150c8" path="/var/lib/kubelet/pods/b8f86227-49ba-4672-864d-1ec158f150c8/volumes" Jul 12 17:55:24.941810 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:24.941780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:24.995651 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:24.995622 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:26.489332 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:26.489302 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:26.587899 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:26.587869 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:55:26.639050 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:55:26.639018 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:01.840155 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:01.840108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:02.046565 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:02.046519 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:04.967880 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:04.967837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:05.072839 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:05.072802 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:05.283948 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:05.283872 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:11.304808 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.304760 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67f56cbff9-8dj2q"] Jul 12 17:56:11.305378 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.305010 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8f86227-49ba-4672-864d-1ec158f150c8" containerName="console" Jul 12 17:56:11.305378 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.305021 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8f86227-49ba-4672-864d-1ec158f150c8" containerName="console" Jul 12 17:56:11.305378 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.305035 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="731149e1-0bb8-440e-a36f-a7f65052bf23" containerName="registry" Jul 12 17:56:11.305378 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.305041 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="731149e1-0bb8-440e-a36f-a7f65052bf23" containerName="registry" Jul 12 17:56:11.305378 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.305101 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="731149e1-0bb8-440e-a36f-a7f65052bf23" containerName="registry" Jul 12 17:56:11.305378 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.305109 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8f86227-49ba-4672-864d-1ec158f150c8" containerName="console" Jul 12 17:56:11.308285 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.308264 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.310896 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.310874 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 12 17:56:11.310896 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.310882 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 12 17:56:11.311070 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.310882 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 12 17:56:11.312032 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.312010 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4rmzj\"" Jul 12 17:56:11.312093 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.312039 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 12 17:56:11.312129 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.312040 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 12 17:56:11.317077 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.317058 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 12 17:56:11.317583 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.317563 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67f56cbff9-8dj2q"] Jul 12 17:56:11.358597 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.358564 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:11.402623 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.402589 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-service-ca\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.402754 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.402630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-trusted-ca-bundle\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.402754 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.402652 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-serving-cert\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.402754 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.402694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-oauth-config\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.402754 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.402723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-oauth-serving-cert\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.402754 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.402748 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-config\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.402914 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.402778 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62dwx\" (UniqueName: \"kubernetes.io/projected/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-kube-api-access-62dwx\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.503787 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.503756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-service-ca\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.503922 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.503797 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-trusted-ca-bundle\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.503922 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.503836 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-serving-cert\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.503922 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.503854 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-oauth-config\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.503922 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.503870 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-oauth-serving-cert\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.503922 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.503892 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-config\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.503922 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.503915 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62dwx\" (UniqueName: \"kubernetes.io/projected/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-kube-api-access-62dwx\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.504581 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.504555 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-service-ca\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.504692 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.504672 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-oauth-serving-cert\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.504829 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.504808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-config\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.504868 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.504843 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-trusted-ca-bundle\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.506370 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.506349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-serving-cert\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.506534 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.506498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-oauth-config\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.511296 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.511279 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62dwx\" (UniqueName: \"kubernetes.io/projected/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-kube-api-access-62dwx\") pod \"console-67f56cbff9-8dj2q\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.617762 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.617676 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:11.755918 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:11.755893 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67f56cbff9-8dj2q"] Jul 12 17:56:11.758227 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:56:11.758199 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c10e35_7d45_4c6c_98e3_29a4e02d8508.slice/crio-c3d71c775c1a608145232e401dda7cae6b2a7d8e79bdc3ee6b81e8f2f66523ba WatchSource:0}: Error finding container c3d71c775c1a608145232e401dda7cae6b2a7d8e79bdc3ee6b81e8f2f66523ba: Status 404 returned error can't find the container with id c3d71c775c1a608145232e401dda7cae6b2a7d8e79bdc3ee6b81e8f2f66523ba Jul 12 17:56:12.582833 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:12.582800 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f56cbff9-8dj2q" event={"ID":"e4c10e35-7d45-4c6c-98e3-29a4e02d8508","Type":"ContainerStarted","Data":"3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4"} Jul 12 17:56:12.583195 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:12.582852 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f56cbff9-8dj2q" event={"ID":"e4c10e35-7d45-4c6c-98e3-29a4e02d8508","Type":"ContainerStarted","Data":"c3d71c775c1a608145232e401dda7cae6b2a7d8e79bdc3ee6b81e8f2f66523ba"} Jul 12 17:56:21.617876 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:21.617796 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:21.617876 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:21.617850 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:21.622410 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:21.622388 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:21.638669 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:21.638627 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67f56cbff9-8dj2q" podStartSLOduration=10.638610732 podStartE2EDuration="10.638610732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:56:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:56:12.600237149 +0000 UTC m=+166.244079871" watchObservedRunningTime="2026-07-12 17:56:21.638610732 +0000 UTC m=+175.282453456" Jul 12 17:56:22.619556 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:22.619530 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:56:22.753459 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:22.753430 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:22.813342 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:22.813310 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:56:22.871487 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:56:22.871399 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:17.059775 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:17.059720 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:17.123334 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:17.123306 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:21.332366 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:21.332334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:26.764123 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.764093 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6ff8f4d8cd-wn7zk"] Jul 12 17:57:26.767150 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.767133 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.778694 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.778672 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff8f4d8cd-wn7zk"] Jul 12 17:57:26.812365 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.812336 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:26.850045 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.850019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-oauth-config\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.850160 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.850050 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-serving-cert\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.850160 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.850112 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-console-config\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.850160 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.850133 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-trusted-ca-bundle\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.850160 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.850155 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-oauth-serving-cert\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.850288 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.850236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qv8v\" (UniqueName: \"kubernetes.io/projected/2756b9c0-3752-4508-970f-16f5c66308fe-kube-api-access-7qv8v\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.850288 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.850262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-service-ca\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.866609 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.866568 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:26.951104 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951054 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-console-config\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.951104 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951108 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-trusted-ca-bundle\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.951273 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-oauth-serving-cert\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.951273 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951266 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qv8v\" (UniqueName: \"kubernetes.io/projected/2756b9c0-3752-4508-970f-16f5c66308fe-kube-api-access-7qv8v\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.951336 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951285 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-service-ca\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.951336 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951323 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-oauth-config\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.951410 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-serving-cert\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.951888 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951866 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-console-config\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.952010 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.951984 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-service-ca\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.952151 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.952061 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-trusted-ca-bundle\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.952283 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.952265 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-oauth-serving-cert\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.953835 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.953812 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-oauth-config\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.953913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.953903 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-serving-cert\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.959036 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.959014 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qv8v\" (UniqueName: \"kubernetes.io/projected/2756b9c0-3752-4508-970f-16f5c66308fe-kube-api-access-7qv8v\") pod \"console-6ff8f4d8cd-wn7zk\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:26.962496 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:26.962476 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:27.013046 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:27.013018 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:27.077556 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:27.077475 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:27.199257 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:27.199223 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ff8f4d8cd-wn7zk"] Jul 12 17:57:27.203557 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:57:27.203533 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2756b9c0_3752_4508_970f_16f5c66308fe.slice/crio-ac5310c58fe7e8cfb9b0494c5b9ea8d5a013906d98bb174e9645d6631c992761 WatchSource:0}: Error finding container ac5310c58fe7e8cfb9b0494c5b9ea8d5a013906d98bb174e9645d6631c992761: Status 404 returned error can't find the container with id ac5310c58fe7e8cfb9b0494c5b9ea8d5a013906d98bb174e9645d6631c992761 Jul 12 17:57:27.589460 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:27.589425 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:27.650341 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:27.650314 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:27.829669 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:27.829639 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff8f4d8cd-wn7zk" event={"ID":"2756b9c0-3752-4508-970f-16f5c66308fe","Type":"ContainerStarted","Data":"99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d"} Jul 12 17:57:27.830046 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:27.829710 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff8f4d8cd-wn7zk" event={"ID":"2756b9c0-3752-4508-970f-16f5c66308fe","Type":"ContainerStarted","Data":"ac5310c58fe7e8cfb9b0494c5b9ea8d5a013906d98bb174e9645d6631c992761"} Jul 12 17:57:27.852248 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:27.852166 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6ff8f4d8cd-wn7zk" podStartSLOduration=1.852152939 podStartE2EDuration="1.852152939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:57:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:57:27.851370036 +0000 UTC m=+241.495212760" watchObservedRunningTime="2026-07-12 17:57:27.852152939 +0000 UTC m=+241.495995664" Jul 12 17:57:29.057956 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:29.057922 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:29.109184 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:29.109151 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:34.564197 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:34.564164 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:35.673752 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:35.673711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:37.078571 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:37.078535 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:37.079101 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:37.078611 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:37.083338 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:37.083318 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:37.866549 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:37.866525 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 17:57:37.914523 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:37.914477 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67f56cbff9-8dj2q"] Jul 12 17:57:38.027864 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:38.027825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:38.127748 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:38.127654 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:57:51.892347 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:57:51.892265 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:02.935309 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:02.935250 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67f56cbff9-8dj2q" podUID="e4c10e35-7d45-4c6c-98e3-29a4e02d8508" containerName="console" containerID="cri-o://3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4" gracePeriod=15 Jul 12 17:58:03.170923 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.170902 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:58:03.200217 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200151 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-service-ca\") pod \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " Jul 12 17:58:03.200217 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200214 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-oauth-config\") pod \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " Jul 12 17:58:03.200399 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200361 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-serving-cert\") pod \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " Jul 12 17:58:03.200456 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200406 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-oauth-serving-cert\") pod \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " Jul 12 17:58:03.200456 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200444 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-trusted-ca-bundle\") pod \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " Jul 12 17:58:03.200574 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200471 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-config\") pod \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " Jul 12 17:58:03.200574 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200527 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62dwx\" (UniqueName: \"kubernetes.io/projected/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-kube-api-access-62dwx\") pod \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\" (UID: \"e4c10e35-7d45-4c6c-98e3-29a4e02d8508\") " Jul 12 17:58:03.200674 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200582 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-service-ca" (OuterVolumeSpecName: "service-ca") pod "e4c10e35-7d45-4c6c-98e3-29a4e02d8508" (UID: "e4c10e35-7d45-4c6c-98e3-29a4e02d8508"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:58:03.200869 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200847 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:03.200976 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200929 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e4c10e35-7d45-4c6c-98e3-29a4e02d8508" (UID: "e4c10e35-7d45-4c6c-98e3-29a4e02d8508"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:58:03.200976 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200926 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e4c10e35-7d45-4c6c-98e3-29a4e02d8508" (UID: "e4c10e35-7d45-4c6c-98e3-29a4e02d8508"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:58:03.201054 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.200972 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-config" (OuterVolumeSpecName: "console-config") pod "e4c10e35-7d45-4c6c-98e3-29a4e02d8508" (UID: "e4c10e35-7d45-4c6c-98e3-29a4e02d8508"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 17:58:03.202295 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.202276 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e4c10e35-7d45-4c6c-98e3-29a4e02d8508" (UID: "e4c10e35-7d45-4c6c-98e3-29a4e02d8508"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:58:03.202531 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.202496 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e4c10e35-7d45-4c6c-98e3-29a4e02d8508" (UID: "e4c10e35-7d45-4c6c-98e3-29a4e02d8508"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 17:58:03.202806 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.202787 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-kube-api-access-62dwx" (OuterVolumeSpecName: "kube-api-access-62dwx") pod "e4c10e35-7d45-4c6c-98e3-29a4e02d8508" (UID: "e4c10e35-7d45-4c6c-98e3-29a4e02d8508"). InnerVolumeSpecName "kube-api-access-62dwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:58:03.301403 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.301373 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-serving-cert\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:03.301403 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.301398 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-oauth-serving-cert\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:03.301403 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.301407 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-trusted-ca-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:03.301618 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.301416 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-config\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:03.301618 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.301426 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62dwx\" (UniqueName: \"kubernetes.io/projected/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-kube-api-access-62dwx\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:03.301618 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.301435 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e4c10e35-7d45-4c6c-98e3-29a4e02d8508-console-oauth-config\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:03.951643 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.951601 2568 generic.go:356] "Generic (PLEG): container finished" podID="e4c10e35-7d45-4c6c-98e3-29a4e02d8508" containerID="3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4" exitCode=0 Jul 12 17:58:03.952033 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.951662 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f56cbff9-8dj2q" Jul 12 17:58:03.952033 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.951682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f56cbff9-8dj2q" event={"ID":"e4c10e35-7d45-4c6c-98e3-29a4e02d8508","Type":"ContainerDied","Data":"3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4"} Jul 12 17:58:03.952033 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.951736 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f56cbff9-8dj2q" event={"ID":"e4c10e35-7d45-4c6c-98e3-29a4e02d8508","Type":"ContainerDied","Data":"c3d71c775c1a608145232e401dda7cae6b2a7d8e79bdc3ee6b81e8f2f66523ba"} Jul 12 17:58:03.952033 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.951757 2568 scope.go:117] "RemoveContainer" containerID="3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4" Jul 12 17:58:03.959913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.959897 2568 scope.go:117] "RemoveContainer" containerID="3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4" Jul 12 17:58:03.960128 ip-10-0-143-136 kubenswrapper[2568]: E0712 17:58:03.960112 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4\": container with ID starting with 3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4 not found: ID does not exist" containerID="3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4" Jul 12 17:58:03.960184 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.960134 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4"} err="failed to get container status \"3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4\": rpc error: code = NotFound desc = could not find container \"3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4\": container with ID starting with 3df0dae5d24b41f0e338cca914f3ff2448ba99ec8112bfcdc0b42ef02b5b86c4 not found: ID does not exist" Jul 12 17:58:03.974684 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.974662 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67f56cbff9-8dj2q"] Jul 12 17:58:03.979218 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:03.979202 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67f56cbff9-8dj2q"] Jul 12 17:58:04.306246 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.306176 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:04.402715 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.402684 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:04.833754 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.833726 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4c10e35-7d45-4c6c-98e3-29a4e02d8508" path="/var/lib/kubelet/pods/e4c10e35-7d45-4c6c-98e3-29a4e02d8508/volumes" Jul 12 17:58:04.929346 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.929318 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd"] Jul 12 17:58:04.929586 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.929573 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4c10e35-7d45-4c6c-98e3-29a4e02d8508" containerName="console" Jul 12 17:58:04.929633 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.929587 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4c10e35-7d45-4c6c-98e3-29a4e02d8508" containerName="console" Jul 12 17:58:04.929668 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.929640 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4c10e35-7d45-4c6c-98e3-29a4e02d8508" containerName="console" Jul 12 17:58:04.933721 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.933704 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:04.936689 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.936671 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 17:58:04.937580 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.937567 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 17:58:04.937634 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.937604 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qjlrd\"" Jul 12 17:58:04.946000 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:04.945978 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd"] Jul 12 17:58:05.013137 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.013106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.013480 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.013154 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.013480 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.013206 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvc4c\" (UniqueName: \"kubernetes.io/projected/4539bed6-4bc5-4b7f-81b1-2a4632220d92-kube-api-access-bvc4c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.114447 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.114370 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.114447 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.114432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvc4c\" (UniqueName: \"kubernetes.io/projected/4539bed6-4bc5-4b7f-81b1-2a4632220d92-kube-api-access-bvc4c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.114619 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.114473 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.114836 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.114810 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.114873 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.114807 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.132874 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.132844 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvc4c\" (UniqueName: \"kubernetes.io/projected/4539bed6-4bc5-4b7f-81b1-2a4632220d92-kube-api-access-bvc4c\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.242739 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.242710 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:05.362554 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.362364 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd"] Jul 12 17:58:05.365188 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:58:05.365137 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4539bed6_4bc5_4b7f_81b1_2a4632220d92.slice/crio-1dd0104e878ec64dfc34f1923773f879e7e3d6ffe8742897282ec99fa3859753 WatchSource:0}: Error finding container 1dd0104e878ec64dfc34f1923773f879e7e3d6ffe8742897282ec99fa3859753: Status 404 returned error can't find the container with id 1dd0104e878ec64dfc34f1923773f879e7e3d6ffe8742897282ec99fa3859753 Jul 12 17:58:05.961108 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:05.961066 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" event={"ID":"4539bed6-4bc5-4b7f-81b1-2a4632220d92","Type":"ContainerStarted","Data":"1dd0104e878ec64dfc34f1923773f879e7e3d6ffe8742897282ec99fa3859753"} Jul 12 17:58:11.985851 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:11.985813 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" event={"ID":"4539bed6-4bc5-4b7f-81b1-2a4632220d92","Type":"ContainerStarted","Data":"2370da767e4193a0dafbdffb8f945b805571d041ad95de1e39faaebb6a485c58"} Jul 12 17:58:12.991152 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:12.991119 2568 generic.go:356] "Generic (PLEG): container finished" podID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerID="2370da767e4193a0dafbdffb8f945b805571d041ad95de1e39faaebb6a485c58" exitCode=0 Jul 12 17:58:12.991529 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:12.991167 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" event={"ID":"4539bed6-4bc5-4b7f-81b1-2a4632220d92","Type":"ContainerDied","Data":"2370da767e4193a0dafbdffb8f945b805571d041ad95de1e39faaebb6a485c58"} Jul 12 17:58:17.006801 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:17.006764 2568 generic.go:356] "Generic (PLEG): container finished" podID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerID="4ae25af5828c049a12bf834f866c75425c6b322a42681b701e496d85893c82dc" exitCode=0 Jul 12 17:58:17.007187 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:17.006840 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" event={"ID":"4539bed6-4bc5-4b7f-81b1-2a4632220d92","Type":"ContainerDied","Data":"4ae25af5828c049a12bf834f866c75425c6b322a42681b701e496d85893c82dc"} Jul 12 17:58:24.035160 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:24.035131 2568 generic.go:356] "Generic (PLEG): container finished" podID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerID="4eddc57a82e180281b0b2c544353df6abcd1e67ab086eb7dc64f7e9347c90d07" exitCode=0 Jul 12 17:58:24.035556 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:24.035221 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" event={"ID":"4539bed6-4bc5-4b7f-81b1-2a4632220d92","Type":"ContainerDied","Data":"4eddc57a82e180281b0b2c544353df6abcd1e67ab086eb7dc64f7e9347c90d07"} Jul 12 17:58:25.215007 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.214977 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:25.266644 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.266617 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-util\") pod \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " Jul 12 17:58:25.266644 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.266645 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-bundle\") pod \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " Jul 12 17:58:25.266862 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.266674 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvc4c\" (UniqueName: \"kubernetes.io/projected/4539bed6-4bc5-4b7f-81b1-2a4632220d92-kube-api-access-bvc4c\") pod \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\" (UID: \"4539bed6-4bc5-4b7f-81b1-2a4632220d92\") " Jul 12 17:58:25.267306 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.267280 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-bundle" (OuterVolumeSpecName: "bundle") pod "4539bed6-4bc5-4b7f-81b1-2a4632220d92" (UID: "4539bed6-4bc5-4b7f-81b1-2a4632220d92"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:58:25.268896 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.268874 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4539bed6-4bc5-4b7f-81b1-2a4632220d92-kube-api-access-bvc4c" (OuterVolumeSpecName: "kube-api-access-bvc4c") pod "4539bed6-4bc5-4b7f-81b1-2a4632220d92" (UID: "4539bed6-4bc5-4b7f-81b1-2a4632220d92"). InnerVolumeSpecName "kube-api-access-bvc4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:58:25.272182 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.272158 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-util" (OuterVolumeSpecName: "util") pod "4539bed6-4bc5-4b7f-81b1-2a4632220d92" (UID: "4539bed6-4bc5-4b7f-81b1-2a4632220d92"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:58:25.367452 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.367366 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:25.367452 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.367396 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvc4c\" (UniqueName: \"kubernetes.io/projected/4539bed6-4bc5-4b7f-81b1-2a4632220d92-kube-api-access-bvc4c\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:25.367452 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:25.367407 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4539bed6-4bc5-4b7f-81b1-2a4632220d92-util\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:26.043785 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:26.043760 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" Jul 12 17:58:26.043970 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:26.043759 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qdtwd" event={"ID":"4539bed6-4bc5-4b7f-81b1-2a4632220d92","Type":"ContainerDied","Data":"1dd0104e878ec64dfc34f1923773f879e7e3d6ffe8742897282ec99fa3859753"} Jul 12 17:58:26.043970 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:26.043901 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dd0104e878ec64dfc34f1923773f879e7e3d6ffe8742897282ec99fa3859753" Jul 12 17:58:26.707986 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:26.707961 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 17:58:30.744013 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:30.743979 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:32.095844 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.095805 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:32.683903 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.683863 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p"] Jul 12 17:58:32.684152 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.684140 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerName="extract" Jul 12 17:58:32.684199 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.684154 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerName="extract" Jul 12 17:58:32.684199 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.684167 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerName="util" Jul 12 17:58:32.684199 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.684172 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerName="util" Jul 12 17:58:32.684199 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.684179 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerName="pull" Jul 12 17:58:32.684199 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.684184 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerName="pull" Jul 12 17:58:32.684340 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.684223 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4539bed6-4bc5-4b7f-81b1-2a4632220d92" containerName="extract" Jul 12 17:58:32.686538 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.686523 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" Jul 12 17:58:32.689136 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.689116 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:58:32.689347 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.689333 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-72h9k\"" Jul 12 17:58:32.689412 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.689379 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 12 17:58:32.700623 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.700602 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p"] Jul 12 17:58:32.822722 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.822688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/519940c0-1441-4176-bf02-76838c3dd4e3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2xs8p\" (UID: \"519940c0-1441-4176-bf02-76838c3dd4e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" Jul 12 17:58:32.822853 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.822737 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgghz\" (UniqueName: \"kubernetes.io/projected/519940c0-1441-4176-bf02-76838c3dd4e3-kube-api-access-tgghz\") pod \"cert-manager-operator-controller-manager-55c444bf86-2xs8p\" (UID: \"519940c0-1441-4176-bf02-76838c3dd4e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" Jul 12 17:58:32.923402 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.923373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/519940c0-1441-4176-bf02-76838c3dd4e3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2xs8p\" (UID: \"519940c0-1441-4176-bf02-76838c3dd4e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" Jul 12 17:58:32.923572 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.923413 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgghz\" (UniqueName: \"kubernetes.io/projected/519940c0-1441-4176-bf02-76838c3dd4e3-kube-api-access-tgghz\") pod \"cert-manager-operator-controller-manager-55c444bf86-2xs8p\" (UID: \"519940c0-1441-4176-bf02-76838c3dd4e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" Jul 12 17:58:32.923772 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.923754 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/519940c0-1441-4176-bf02-76838c3dd4e3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2xs8p\" (UID: \"519940c0-1441-4176-bf02-76838c3dd4e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" Jul 12 17:58:32.935438 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.935380 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgghz\" (UniqueName: \"kubernetes.io/projected/519940c0-1441-4176-bf02-76838c3dd4e3-kube-api-access-tgghz\") pod \"cert-manager-operator-controller-manager-55c444bf86-2xs8p\" (UID: \"519940c0-1441-4176-bf02-76838c3dd4e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" Jul 12 17:58:32.994924 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:32.994897 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" Jul 12 17:58:33.123230 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:33.123209 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p"] Jul 12 17:58:33.125741 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:58:33.125716 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519940c0_1441_4176_bf02_76838c3dd4e3.slice/crio-463ae3b2e59aceab8269c73c01313e25aba78fecd6f6fb88308c124fbeb73c6e WatchSource:0}: Error finding container 463ae3b2e59aceab8269c73c01313e25aba78fecd6f6fb88308c124fbeb73c6e: Status 404 returned error can't find the container with id 463ae3b2e59aceab8269c73c01313e25aba78fecd6f6fb88308c124fbeb73c6e Jul 12 17:58:33.128368 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:33.128350 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 17:58:34.074161 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:34.074116 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" event={"ID":"519940c0-1441-4176-bf02-76838c3dd4e3","Type":"ContainerStarted","Data":"463ae3b2e59aceab8269c73c01313e25aba78fecd6f6fb88308c124fbeb73c6e"} Jul 12 17:58:34.353061 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:34.352994 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:34.453613 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:34.453576 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:35.080255 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:35.080172 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" event={"ID":"519940c0-1441-4176-bf02-76838c3dd4e3","Type":"ContainerStarted","Data":"1daa5d990dec3562435c4a3a9b4c5fb27c11bd21a7c4c25a528a477feb4b458f"} Jul 12 17:58:35.101749 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:35.101706 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2xs8p" podStartSLOduration=1.447867972 podStartE2EDuration="3.101690816s" totalImagesPullingTime="1.653822844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:58:35.09916268 +0000 UTC m=+308.743005407" watchObservedRunningTime="2026-07-12 17:58:35.101690816 +0000 UTC m=+308.745533538" Jul 12 17:58:35.443604 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:35.443578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:35.549248 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:35.549211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:36.385366 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.385329 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl"] Jul 12 17:58:36.388874 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.388857 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.391723 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.391700 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 17:58:36.391840 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.391704 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 17:58:36.392469 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.392452 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qjlrd\"" Jul 12 17:58:36.397710 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.397686 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl"] Jul 12 17:58:36.453827 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.453795 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.454173 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.453838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2znb\" (UniqueName: \"kubernetes.io/projected/3ca761fb-7629-4254-995c-ec89adea623b-kube-api-access-q2znb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.454173 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.453920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.555043 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.555012 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.555181 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.555052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.555181 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.555076 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2znb\" (UniqueName: \"kubernetes.io/projected/3ca761fb-7629-4254-995c-ec89adea623b-kube-api-access-q2znb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.555377 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.555358 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.555412 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.555399 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.568711 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.568684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2znb\" (UniqueName: \"kubernetes.io/projected/3ca761fb-7629-4254-995c-ec89adea623b-kube-api-access-q2znb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.698911 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.698879 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:36.822404 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:36.822335 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl"] Jul 12 17:58:36.825100 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:58:36.825071 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca761fb_7629_4254_995c_ec89adea623b.slice/crio-57d26ae69ee4dde09365ef4e1e3ad84e98cdb7e6e20a2abd5de8bbfcf7422c4b WatchSource:0}: Error finding container 57d26ae69ee4dde09365ef4e1e3ad84e98cdb7e6e20a2abd5de8bbfcf7422c4b: Status 404 returned error can't find the container with id 57d26ae69ee4dde09365ef4e1e3ad84e98cdb7e6e20a2abd5de8bbfcf7422c4b Jul 12 17:58:37.088271 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.088194 2568 generic.go:356] "Generic (PLEG): container finished" podID="3ca761fb-7629-4254-995c-ec89adea623b" containerID="aeec5b9f8a8b779b8014cdfb2a0b3dbca3246424ec3029f95f7d4b4e96ae4180" exitCode=0 Jul 12 17:58:37.088271 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.088239 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" event={"ID":"3ca761fb-7629-4254-995c-ec89adea623b","Type":"ContainerDied","Data":"aeec5b9f8a8b779b8014cdfb2a0b3dbca3246424ec3029f95f7d4b4e96ae4180"} Jul 12 17:58:37.088450 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.088279 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" event={"ID":"3ca761fb-7629-4254-995c-ec89adea623b","Type":"ContainerStarted","Data":"57d26ae69ee4dde09365ef4e1e3ad84e98cdb7e6e20a2abd5de8bbfcf7422c4b"} Jul 12 17:58:37.256764 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.256731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:37.745432 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.745374 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:37.799156 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.799124 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt"] Jul 12 17:58:37.802660 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.802589 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" Jul 12 17:58:37.804981 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.804961 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 17:58:37.806138 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.806077 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 17:58:37.806415 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.806399 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-c7f86\"" Jul 12 17:58:37.808617 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.808575 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt"] Jul 12 17:58:37.866286 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.866197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273c4681-0625-4eb2-b234-501c1bbec1af-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rq4bt\" (UID: \"273c4681-0625-4eb2-b234-501c1bbec1af\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" Jul 12 17:58:37.866444 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.866288 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxx8d\" (UniqueName: \"kubernetes.io/projected/273c4681-0625-4eb2-b234-501c1bbec1af-kube-api-access-sxx8d\") pod \"cert-manager-cainjector-559cdd8c87-rq4bt\" (UID: \"273c4681-0625-4eb2-b234-501c1bbec1af\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" Jul 12 17:58:37.966976 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.966944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxx8d\" (UniqueName: \"kubernetes.io/projected/273c4681-0625-4eb2-b234-501c1bbec1af-kube-api-access-sxx8d\") pod \"cert-manager-cainjector-559cdd8c87-rq4bt\" (UID: \"273c4681-0625-4eb2-b234-501c1bbec1af\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" Jul 12 17:58:37.967152 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.967011 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273c4681-0625-4eb2-b234-501c1bbec1af-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rq4bt\" (UID: \"273c4681-0625-4eb2-b234-501c1bbec1af\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" Jul 12 17:58:37.976031 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.975977 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273c4681-0625-4eb2-b234-501c1bbec1af-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rq4bt\" (UID: \"273c4681-0625-4eb2-b234-501c1bbec1af\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" Jul 12 17:58:37.976215 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:37.976085 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxx8d\" (UniqueName: \"kubernetes.io/projected/273c4681-0625-4eb2-b234-501c1bbec1af-kube-api-access-sxx8d\") pod \"cert-manager-cainjector-559cdd8c87-rq4bt\" (UID: \"273c4681-0625-4eb2-b234-501c1bbec1af\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" Jul 12 17:58:38.120553 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:38.120466 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" Jul 12 17:58:38.254740 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:38.254717 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt"] Jul 12 17:58:38.256785 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:58:38.256755 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod273c4681_0625_4eb2_b234_501c1bbec1af.slice/crio-87da0bdc9cd15f3c5762a4d3b1ed39fda24b60489783a3109586065ab77ec544 WatchSource:0}: Error finding container 87da0bdc9cd15f3c5762a4d3b1ed39fda24b60489783a3109586065ab77ec544: Status 404 returned error can't find the container with id 87da0bdc9cd15f3c5762a4d3b1ed39fda24b60489783a3109586065ab77ec544 Jul 12 17:58:39.102469 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:39.102437 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" event={"ID":"273c4681-0625-4eb2-b234-501c1bbec1af","Type":"ContainerStarted","Data":"87da0bdc9cd15f3c5762a4d3b1ed39fda24b60489783a3109586065ab77ec544"} Jul 12 17:58:39.104247 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:39.104218 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" event={"ID":"3ca761fb-7629-4254-995c-ec89adea623b","Type":"ContainerStarted","Data":"5d1a0639327916d0fe5528b49569f7261312655ecbd59cf2413a21930966948e"} Jul 12 17:58:40.111455 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:40.111418 2568 generic.go:356] "Generic (PLEG): container finished" podID="3ca761fb-7629-4254-995c-ec89adea623b" containerID="5d1a0639327916d0fe5528b49569f7261312655ecbd59cf2413a21930966948e" exitCode=0 Jul 12 17:58:40.111914 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:40.111496 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" event={"ID":"3ca761fb-7629-4254-995c-ec89adea623b","Type":"ContainerDied","Data":"5d1a0639327916d0fe5528b49569f7261312655ecbd59cf2413a21930966948e"} Jul 12 17:58:41.117782 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.117747 2568 generic.go:356] "Generic (PLEG): container finished" podID="3ca761fb-7629-4254-995c-ec89adea623b" containerID="d79772349afd5f7ebe586ec39793e72639e5713756c1194d545e19a5273e3e91" exitCode=0 Jul 12 17:58:41.118190 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.117843 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" event={"ID":"3ca761fb-7629-4254-995c-ec89adea623b","Type":"ContainerDied","Data":"d79772349afd5f7ebe586ec39793e72639e5713756c1194d545e19a5273e3e91"} Jul 12 17:58:41.363470 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.363364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:41.418044 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.418005 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:41.516528 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.516480 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mzlx2"] Jul 12 17:58:41.519478 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.519460 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:41.521816 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.521798 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vhvc5\"" Jul 12 17:58:41.529021 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.529003 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mzlx2"] Jul 12 17:58:41.598105 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.598070 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/439afd77-9b92-4449-a14e-7a8bb07eb489-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mzlx2\" (UID: \"439afd77-9b92-4449-a14e-7a8bb07eb489\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:41.598270 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.598112 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mww5\" (UniqueName: \"kubernetes.io/projected/439afd77-9b92-4449-a14e-7a8bb07eb489-kube-api-access-4mww5\") pod \"cert-manager-webhook-6486ffd997-mzlx2\" (UID: \"439afd77-9b92-4449-a14e-7a8bb07eb489\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:41.698979 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.698942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mww5\" (UniqueName: \"kubernetes.io/projected/439afd77-9b92-4449-a14e-7a8bb07eb489-kube-api-access-4mww5\") pod \"cert-manager-webhook-6486ffd997-mzlx2\" (UID: \"439afd77-9b92-4449-a14e-7a8bb07eb489\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:41.699172 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.699036 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/439afd77-9b92-4449-a14e-7a8bb07eb489-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mzlx2\" (UID: \"439afd77-9b92-4449-a14e-7a8bb07eb489\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:41.708030 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.707994 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/439afd77-9b92-4449-a14e-7a8bb07eb489-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mzlx2\" (UID: \"439afd77-9b92-4449-a14e-7a8bb07eb489\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:41.708149 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.708134 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mww5\" (UniqueName: \"kubernetes.io/projected/439afd77-9b92-4449-a14e-7a8bb07eb489-kube-api-access-4mww5\") pod \"cert-manager-webhook-6486ffd997-mzlx2\" (UID: \"439afd77-9b92-4449-a14e-7a8bb07eb489\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:41.828913 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.828882 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:41.958275 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:41.958254 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mzlx2"] Jul 12 17:58:41.960600 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:58:41.960570 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod439afd77_9b92_4449_a14e_7a8bb07eb489.slice/crio-f8258df18efb99a5729de36f7eec7b04c3c52f49337cb0748fa002774de2f21c WatchSource:0}: Error finding container f8258df18efb99a5729de36f7eec7b04c3c52f49337cb0748fa002774de2f21c: Status 404 returned error can't find the container with id f8258df18efb99a5729de36f7eec7b04c3c52f49337cb0748fa002774de2f21c Jul 12 17:58:42.124270 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.124227 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" event={"ID":"273c4681-0625-4eb2-b234-501c1bbec1af","Type":"ContainerStarted","Data":"d941e73f37fc198d182bc4d774012457375c361654f840237e7121d66079aea6"} Jul 12 17:58:42.125809 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.125784 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" event={"ID":"439afd77-9b92-4449-a14e-7a8bb07eb489","Type":"ContainerStarted","Data":"53fa9e05169664b8db055d11c00c025f78b6654e4f20ec4d0a82424284ea6bb7"} Jul 12 17:58:42.125911 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.125821 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" event={"ID":"439afd77-9b92-4449-a14e-7a8bb07eb489","Type":"ContainerStarted","Data":"f8258df18efb99a5729de36f7eec7b04c3c52f49337cb0748fa002774de2f21c"} Jul 12 17:58:42.125966 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.125949 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:42.143857 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.143801 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rq4bt" podStartSLOduration=2.260755176 podStartE2EDuration="5.143785847s" totalImagesPullingTime="2.883030671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:58:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:58:42.142764987 +0000 UTC m=+315.786607711" watchObservedRunningTime="2026-07-12 17:58:42.143785847 +0000 UTC m=+315.787628570" Jul 12 17:58:42.169490 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.169438 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" podStartSLOduration=1.169418041 podStartE2EDuration="1.169418041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:58:42.167947346 +0000 UTC m=+315.811790052" watchObservedRunningTime="2026-07-12 17:58:42.169418041 +0000 UTC m=+315.813260763" Jul 12 17:58:42.218348 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.218227 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:42.268882 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.268859 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:42.405453 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.405414 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2znb\" (UniqueName: \"kubernetes.io/projected/3ca761fb-7629-4254-995c-ec89adea623b-kube-api-access-q2znb\") pod \"3ca761fb-7629-4254-995c-ec89adea623b\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " Jul 12 17:58:42.405453 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.405464 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-bundle\") pod \"3ca761fb-7629-4254-995c-ec89adea623b\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " Jul 12 17:58:42.405698 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.405487 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-util\") pod \"3ca761fb-7629-4254-995c-ec89adea623b\" (UID: \"3ca761fb-7629-4254-995c-ec89adea623b\") " Jul 12 17:58:42.405849 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.405827 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-bundle" (OuterVolumeSpecName: "bundle") pod "3ca761fb-7629-4254-995c-ec89adea623b" (UID: "3ca761fb-7629-4254-995c-ec89adea623b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:58:42.407601 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.407575 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ca761fb-7629-4254-995c-ec89adea623b-kube-api-access-q2znb" (OuterVolumeSpecName: "kube-api-access-q2znb") pod "3ca761fb-7629-4254-995c-ec89adea623b" (UID: "3ca761fb-7629-4254-995c-ec89adea623b"). InnerVolumeSpecName "kube-api-access-q2znb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:58:42.409903 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.409880 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-util" (OuterVolumeSpecName: "util") pod "3ca761fb-7629-4254-995c-ec89adea623b" (UID: "3ca761fb-7629-4254-995c-ec89adea623b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:58:42.506838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.506752 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-util\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:42.506838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.506778 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2znb\" (UniqueName: \"kubernetes.io/projected/3ca761fb-7629-4254-995c-ec89adea623b-kube-api-access-q2znb\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:42.506838 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:42.506788 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3ca761fb-7629-4254-995c-ec89adea623b-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:43.131864 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:43.131789 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" Jul 12 17:58:43.131864 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:43.131824 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7ksjl" event={"ID":"3ca761fb-7629-4254-995c-ec89adea623b","Type":"ContainerDied","Data":"57d26ae69ee4dde09365ef4e1e3ad84e98cdb7e6e20a2abd5de8bbfcf7422c4b"} Jul 12 17:58:43.132302 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:43.131916 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57d26ae69ee4dde09365ef4e1e3ad84e98cdb7e6e20a2abd5de8bbfcf7422c4b" Jul 12 17:58:44.982992 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:44.982957 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:45.067356 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:45.067325 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:46.553551 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:46.553524 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:48.134451 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:48.134423 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mzlx2" Jul 12 17:58:53.611756 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:53.611715 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:53.709481 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:53.709428 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:55.220068 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.220031 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw"] Jul 12 17:58:55.221539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.220951 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca761fb-7629-4254-995c-ec89adea623b" containerName="pull" Jul 12 17:58:55.221539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.220991 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca761fb-7629-4254-995c-ec89adea623b" containerName="pull" Jul 12 17:58:55.221539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.221003 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca761fb-7629-4254-995c-ec89adea623b" containerName="extract" Jul 12 17:58:55.221539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.221012 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca761fb-7629-4254-995c-ec89adea623b" containerName="extract" Jul 12 17:58:55.221539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.221037 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca761fb-7629-4254-995c-ec89adea623b" containerName="util" Jul 12 17:58:55.221539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.221053 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca761fb-7629-4254-995c-ec89adea623b" containerName="util" Jul 12 17:58:55.221539 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.221218 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca761fb-7629-4254-995c-ec89adea623b" containerName="extract" Jul 12 17:58:55.227327 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.227302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.229850 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.229817 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 17:58:55.230266 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.230245 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw"] Jul 12 17:58:55.230738 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.230716 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qjlrd\"" Jul 12 17:58:55.230853 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.230728 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 17:58:55.326148 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.326115 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:58:55.407366 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.407331 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.407556 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.407380 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.407556 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.407457 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9dhz\" (UniqueName: \"kubernetes.io/projected/28d0b6d0-4e44-4544-9c72-2b207b83607c-kube-api-access-g9dhz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.508373 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.508275 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9dhz\" (UniqueName: \"kubernetes.io/projected/28d0b6d0-4e44-4544-9c72-2b207b83607c-kube-api-access-g9dhz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.508373 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.508350 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.508638 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.508389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.508764 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.508741 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.508825 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.508785 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.515807 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.515782 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9dhz\" (UniqueName: \"kubernetes.io/projected/28d0b6d0-4e44-4544-9c72-2b207b83607c-kube-api-access-g9dhz\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.537765 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.537730 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:55.662918 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:55.662896 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw"] Jul 12 17:58:55.664830 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:58:55.664800 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28d0b6d0_4e44_4544_9c72_2b207b83607c.slice/crio-a94296624eebfd43e15d55c0efc0dc3c049fae88330bd4cd1dca1c7315017ca1 WatchSource:0}: Error finding container a94296624eebfd43e15d55c0efc0dc3c049fae88330bd4cd1dca1c7315017ca1: Status 404 returned error can't find the container with id a94296624eebfd43e15d55c0efc0dc3c049fae88330bd4cd1dca1c7315017ca1 Jul 12 17:58:56.186174 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:56.186145 2568 generic.go:356] "Generic (PLEG): container finished" podID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerID="385da99a84187412296c6375729b979e681447cf39367aab917cf2a6cf67c31e" exitCode=0 Jul 12 17:58:56.186340 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:56.186230 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" event={"ID":"28d0b6d0-4e44-4544-9c72-2b207b83607c","Type":"ContainerDied","Data":"385da99a84187412296c6375729b979e681447cf39367aab917cf2a6cf67c31e"} Jul 12 17:58:56.186340 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:56.186276 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" event={"ID":"28d0b6d0-4e44-4544-9c72-2b207b83607c","Type":"ContainerStarted","Data":"a94296624eebfd43e15d55c0efc0dc3c049fae88330bd4cd1dca1c7315017ca1"} Jul 12 17:58:57.191759 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:57.191728 2568 generic.go:356] "Generic (PLEG): container finished" podID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerID="c8855ace55d28992fc8610b68260ee32f5442362f4b39f744f52bc58b7f9a3ef" exitCode=0 Jul 12 17:58:57.192257 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:57.191793 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" event={"ID":"28d0b6d0-4e44-4544-9c72-2b207b83607c","Type":"ContainerDied","Data":"c8855ace55d28992fc8610b68260ee32f5442362f4b39f744f52bc58b7f9a3ef"} Jul 12 17:58:58.198005 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:58.197968 2568 generic.go:356] "Generic (PLEG): container finished" podID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerID="a73bb209d73347f33d63a19fc1fc53748c6364985c8899f2170d4a899f2454bf" exitCode=0 Jul 12 17:58:58.198342 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:58.198047 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" event={"ID":"28d0b6d0-4e44-4544-9c72-2b207b83607c","Type":"ContainerDied","Data":"a73bb209d73347f33d63a19fc1fc53748c6364985c8899f2170d4a899f2454bf"} Jul 12 17:58:59.328290 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.328263 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:58:59.442412 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.442376 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-bundle\") pod \"28d0b6d0-4e44-4544-9c72-2b207b83607c\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " Jul 12 17:58:59.442631 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.442445 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9dhz\" (UniqueName: \"kubernetes.io/projected/28d0b6d0-4e44-4544-9c72-2b207b83607c-kube-api-access-g9dhz\") pod \"28d0b6d0-4e44-4544-9c72-2b207b83607c\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " Jul 12 17:58:59.442631 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.442491 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-util\") pod \"28d0b6d0-4e44-4544-9c72-2b207b83607c\" (UID: \"28d0b6d0-4e44-4544-9c72-2b207b83607c\") " Jul 12 17:58:59.443123 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.443090 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-bundle" (OuterVolumeSpecName: "bundle") pod "28d0b6d0-4e44-4544-9c72-2b207b83607c" (UID: "28d0b6d0-4e44-4544-9c72-2b207b83607c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:58:59.444543 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.444500 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28d0b6d0-4e44-4544-9c72-2b207b83607c-kube-api-access-g9dhz" (OuterVolumeSpecName: "kube-api-access-g9dhz") pod "28d0b6d0-4e44-4544-9c72-2b207b83607c" (UID: "28d0b6d0-4e44-4544-9c72-2b207b83607c"). InnerVolumeSpecName "kube-api-access-g9dhz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 17:58:59.448002 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.447969 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-util" (OuterVolumeSpecName: "util") pod "28d0b6d0-4e44-4544-9c72-2b207b83607c" (UID: "28d0b6d0-4e44-4544-9c72-2b207b83607c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 17:58:59.544082 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.543995 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:59.544082 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.544027 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9dhz\" (UniqueName: \"kubernetes.io/projected/28d0b6d0-4e44-4544-9c72-2b207b83607c-kube-api-access-g9dhz\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:58:59.544082 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:58:59.544039 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/28d0b6d0-4e44-4544-9c72-2b207b83607c-util\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 17:59:00.212420 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:00.212387 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" event={"ID":"28d0b6d0-4e44-4544-9c72-2b207b83607c","Type":"ContainerDied","Data":"a94296624eebfd43e15d55c0efc0dc3c049fae88330bd4cd1dca1c7315017ca1"} Jul 12 17:59:00.212420 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:00.212419 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxwjxw" Jul 12 17:59:00.212689 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:00.212434 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a94296624eebfd43e15d55c0efc0dc3c049fae88330bd4cd1dca1c7315017ca1" Jul 12 17:59:11.364661 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.364625 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:11.458985 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.458956 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:11.594149 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.594114 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:11.668293 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.668260 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk"] Jul 12 17:59:11.668568 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.668555 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerName="extract" Jul 12 17:59:11.668616 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.668569 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerName="extract" Jul 12 17:59:11.668616 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.668588 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerName="pull" Jul 12 17:59:11.668616 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.668593 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerName="pull" Jul 12 17:59:11.668616 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.668601 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerName="util" Jul 12 17:59:11.668616 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.668607 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerName="util" Jul 12 17:59:11.668771 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.668653 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="28d0b6d0-4e44-4544-9c72-2b207b83607c" containerName="extract" Jul 12 17:59:11.671701 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.671678 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.674359 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.674334 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 17:59:11.674632 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.674616 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2kpkp\"" Jul 12 17:59:11.675658 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.675632 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 17:59:11.675756 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.675633 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 17:59:11.675756 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.675688 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 17:59:11.675756 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.675689 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 17:59:11.680037 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.680016 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk"] Jul 12 17:59:11.691344 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.691321 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:11.732701 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.732667 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3aadf229-7793-435d-a708-451cb010f705-metrics-cert\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.732835 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.732714 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3aadf229-7793-435d-a708-451cb010f705-manager-config\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.732835 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.732783 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks4gf\" (UniqueName: \"kubernetes.io/projected/3aadf229-7793-435d-a708-451cb010f705-kube-api-access-ks4gf\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.732835 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.732811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3aadf229-7793-435d-a708-451cb010f705-cert\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.833256 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.833223 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3aadf229-7793-435d-a708-451cb010f705-manager-config\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.833416 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.833277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks4gf\" (UniqueName: \"kubernetes.io/projected/3aadf229-7793-435d-a708-451cb010f705-kube-api-access-ks4gf\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.833416 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.833296 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3aadf229-7793-435d-a708-451cb010f705-cert\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.833416 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.833321 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3aadf229-7793-435d-a708-451cb010f705-metrics-cert\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.833873 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.833853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3aadf229-7793-435d-a708-451cb010f705-manager-config\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.835922 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.835890 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3aadf229-7793-435d-a708-451cb010f705-cert\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.835973 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.835950 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3aadf229-7793-435d-a708-451cb010f705-metrics-cert\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.841587 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.841561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks4gf\" (UniqueName: \"kubernetes.io/projected/3aadf229-7793-435d-a708-451cb010f705-kube-api-access-ks4gf\") pod \"lws-controller-manager-5484cf687f-dl8jk\" (UID: \"3aadf229-7793-435d-a708-451cb010f705\") " pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:11.982027 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:11.981945 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:12.012907 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.012874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:12.107540 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.107496 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk"] Jul 12 17:59:12.109521 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:59:12.109481 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aadf229_7793_435d_a708_451cb010f705.slice/crio-2b5623b6094a4d5a6c7e000c60b1309954a7e024a919d7b7351f92f33b971d82 WatchSource:0}: Error finding container 2b5623b6094a4d5a6c7e000c60b1309954a7e024a919d7b7351f92f33b971d82: Status 404 returned error can't find the container with id 2b5623b6094a4d5a6c7e000c60b1309954a7e024a919d7b7351f92f33b971d82 Jul 12 17:59:12.234053 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.233973 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:12.261355 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.261316 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" event={"ID":"3aadf229-7793-435d-a708-451cb010f705","Type":"ContainerStarted","Data":"2b5623b6094a4d5a6c7e000c60b1309954a7e024a919d7b7351f92f33b971d82"} Jul 12 17:59:12.334228 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.334196 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:12.373441 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.373410 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5"] Jul 12 17:59:12.377749 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.377729 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.380543 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.380503 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 17:59:12.380637 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.380540 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 17:59:12.380637 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.380569 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 17:59:12.380950 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.380936 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m5hgb\"" Jul 12 17:59:12.381023 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.380939 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 17:59:12.405447 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.405423 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5"] Jul 12 17:59:12.437316 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.437282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84be2600-f092-4624-a397-0192870543db-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.437462 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.437325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p42bd\" (UniqueName: \"kubernetes.io/projected/84be2600-f092-4624-a397-0192870543db-kube-api-access-p42bd\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.437462 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.437356 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/84be2600-f092-4624-a397-0192870543db-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.437462 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.437371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84be2600-f092-4624-a397-0192870543db-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.437462 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.437399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/84be2600-f092-4624-a397-0192870543db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.537989 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.537886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84be2600-f092-4624-a397-0192870543db-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.538163 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.537987 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p42bd\" (UniqueName: \"kubernetes.io/projected/84be2600-f092-4624-a397-0192870543db-kube-api-access-p42bd\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.538163 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.538042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/84be2600-f092-4624-a397-0192870543db-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.538163 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.538067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84be2600-f092-4624-a397-0192870543db-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.538163 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.538092 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/84be2600-f092-4624-a397-0192870543db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.538659 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.538632 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/84be2600-f092-4624-a397-0192870543db-manifests\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.540890 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.540864 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84be2600-f092-4624-a397-0192870543db-tmp\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.541005 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.540919 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84be2600-f092-4624-a397-0192870543db-webhook-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.541005 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.540921 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/84be2600-f092-4624-a397-0192870543db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.546866 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.546842 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p42bd\" (UniqueName: \"kubernetes.io/projected/84be2600-f092-4624-a397-0192870543db-kube-api-access-p42bd\") pod \"opendatahub-operator-controller-manager-5dbd4cc77c-rnps5\" (UID: \"84be2600-f092-4624-a397-0192870543db\") " pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.576930 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.576887 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:12.677738 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.677679 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:12.687760 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.687739 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:12.776986 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.776960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:12.821168 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.821133 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5"] Jul 12 17:59:12.823380 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:59:12.823352 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84be2600_f092_4624_a397_0192870543db.slice/crio-1b8120083e56564849c79da57fa8435878122804bca8d69e39d4772d5da4a6a1 WatchSource:0}: Error finding container 1b8120083e56564849c79da57fa8435878122804bca8d69e39d4772d5da4a6a1: Status 404 returned error can't find the container with id 1b8120083e56564849c79da57fa8435878122804bca8d69e39d4772d5da4a6a1 Jul 12 17:59:12.876900 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.876868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:12.977753 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:12.977724 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:13.086254 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:13.086164 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:13.267725 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:13.267691 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" event={"ID":"84be2600-f092-4624-a397-0192870543db","Type":"ContainerStarted","Data":"1b8120083e56564849c79da57fa8435878122804bca8d69e39d4772d5da4a6a1"} Jul 12 17:59:14.273862 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:14.273748 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" event={"ID":"3aadf229-7793-435d-a708-451cb010f705","Type":"ContainerStarted","Data":"900585e8339519e15d66e7c69db54196f355102397f3035396fba8c70e341ac1"} Jul 12 17:59:14.274389 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:14.273871 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:14.290452 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:14.290394 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" podStartSLOduration=1.7996657 podStartE2EDuration="3.290371488s" totalImagesPullingTime="1.490705788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:14.289932893 +0000 UTC m=+347.933775616" watchObservedRunningTime="2026-07-12 17:59:14.290371488 +0000 UTC m=+347.934214212" Jul 12 17:59:16.284002 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:16.283909 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" event={"ID":"84be2600-f092-4624-a397-0192870543db","Type":"ContainerStarted","Data":"1291743700bb25f66d103da2ce414e05035f4325151da1faa7963ae5e9f49527"} Jul 12 17:59:17.289650 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:17.289611 2568 generic.go:356] "Generic (PLEG): container finished" podID="84be2600-f092-4624-a397-0192870543db" containerID="1291743700bb25f66d103da2ce414e05035f4325151da1faa7963ae5e9f49527" exitCode=0 Jul 12 17:59:17.290032 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:17.289693 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" event={"ID":"84be2600-f092-4624-a397-0192870543db","Type":"ContainerDied","Data":"1291743700bb25f66d103da2ce414e05035f4325151da1faa7963ae5e9f49527"} Jul 12 17:59:18.295570 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:18.295534 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" event={"ID":"84be2600-f092-4624-a397-0192870543db","Type":"ContainerStarted","Data":"8974b98096d35dcb824379c0a18a54cf654ae6cbebca760965ca6e5545bfd1ed"} Jul 12 17:59:18.295968 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:18.295668 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:18.316998 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:18.316946 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" podStartSLOduration=1.8020213809999999 podStartE2EDuration="6.316930502s" totalImagesPullingTime="3.479462664s" totalInitContainerRuntime="1.035446457s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:18.315009517 +0000 UTC m=+351.958852265" watchObservedRunningTime="2026-07-12 17:59:18.316930502 +0000 UTC m=+351.960773264" Jul 12 17:59:24.132745 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:24.132689 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:24.224229 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:24.224191 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:25.281063 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:25.281034 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5484cf687f-dl8jk" Jul 12 17:59:29.301750 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:29.301720 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5dbd4cc77c-rnps5" Jul 12 17:59:31.439766 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:31.439730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:31.556247 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:31.556212 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:31.920903 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:31.920847 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:32.209295 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:32.209209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:32.362556 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:32.362519 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:32.462228 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:32.462117 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:32.538314 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:32.538286 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:32.689077 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:32.689047 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:33.237236 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:33.237202 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:33.366253 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:33.366220 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:33.437178 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:33.437144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:34.282958 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:34.282925 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:38.849333 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:38.849296 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:38.939082 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:38.939050 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:39.281636 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.281603 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s"] Jul 12 17:59:39.294118 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.294095 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.297736 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.297713 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 12 17:59:39.297858 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.297823 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 12 17:59:39.301540 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.301497 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 12 17:59:39.301822 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.301793 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7rjg7\"" Jul 12 17:59:39.302053 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.302010 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 17:59:39.317693 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.317662 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s"] Jul 12 17:59:39.361579 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.361544 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.361765 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.361596 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfj28\" (UniqueName: \"kubernetes.io/projected/953be662-d153-46f3-ab5f-375a5eba7a2f-kube-api-access-cfj28\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.361765 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.361674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.361765 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.361727 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.361765 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.361758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.361940 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.361790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.361940 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.361820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/953be662-d153-46f3-ab5f-375a5eba7a2f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.462592 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.462561 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.462774 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.462610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/953be662-d153-46f3-ab5f-375a5eba7a2f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.462774 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.462644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.462774 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.462674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfj28\" (UniqueName: \"kubernetes.io/projected/953be662-d153-46f3-ab5f-375a5eba7a2f-kube-api-access-cfj28\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.462774 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.462740 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.462774 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.462772 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.463039 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.462797 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.463550 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.463525 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.465061 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.465040 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/953be662-d153-46f3-ab5f-375a5eba7a2f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.465252 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.465229 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.465324 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.465262 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.465361 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.465346 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.480102 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.480076 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/953be662-d153-46f3-ab5f-375a5eba7a2f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.480209 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.480190 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfj28\" (UniqueName: \"kubernetes.io/projected/953be662-d153-46f3-ab5f-375a5eba7a2f-kube-api-access-cfj28\") pod \"istiod-openshift-gateway-94bb8fbfd-7jm6s\" (UID: \"953be662-d153-46f3-ab5f-375a5eba7a2f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.494808 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.494782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:39.604219 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.604135 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:39.704335 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.704300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:39.773940 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.773915 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s"] Jul 12 17:59:39.774888 ip-10-0-143-136 kubenswrapper[2568]: W0712 17:59:39.774862 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953be662_d153_46f3_ab5f_375a5eba7a2f.slice/crio-1ed8e13cb4bc8a22b38327fa542f3b0c138dcc38c98de8a6ca0c5b2782415465 WatchSource:0}: Error finding container 1ed8e13cb4bc8a22b38327fa542f3b0c138dcc38c98de8a6ca0c5b2782415465: Status 404 returned error can't find the container with id 1ed8e13cb4bc8a22b38327fa542f3b0c138dcc38c98de8a6ca0c5b2782415465 Jul 12 17:59:39.820947 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:39.820910 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:40.391717 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:40.391329 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" event={"ID":"953be662-d153-46f3-ab5f-375a5eba7a2f","Type":"ContainerStarted","Data":"1ed8e13cb4bc8a22b38327fa542f3b0c138dcc38c98de8a6ca0c5b2782415465"} Jul 12 17:59:41.919438 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:41.919402 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:42.028041 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:42.027975 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:42.145784 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:42.145748 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 17:59:42.145910 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:42.145836 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 12 17:59:42.217709 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:42.217672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:42.325928 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:42.325896 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:42.405612 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:42.405583 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" event={"ID":"953be662-d153-46f3-ab5f-375a5eba7a2f","Type":"ContainerStarted","Data":"9cce010b35f7d751e34f550c66d204fe7f938ac33d63f45ac5a4b818fdfbfd5c"} Jul 12 17:59:42.405795 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:42.405707 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 17:59:42.429139 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:42.429091 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" podStartSLOduration=1.060717893 podStartE2EDuration="3.429075111s" totalImagesPullingTime="2.368357218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 17:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 17:59:42.427452376 +0000 UTC m=+376.071295136" watchObservedRunningTime="2026-07-12 17:59:42.429075111 +0000 UTC m=+376.072917833" Jul 12 17:59:43.201630 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:43.201596 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:43.304087 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:43.304048 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 17:59:43.411769 ip-10-0-143-136 kubenswrapper[2568]: I0712 17:59:43.411744 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-7jm6s" Jul 12 18:00:10.474432 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:10.474399 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:10.525155 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:10.525126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:10.574622 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:10.574594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:12.215756 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:12.215721 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:12.268196 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:12.268140 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:29.407392 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:29.407354 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:29.458434 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:29.458400 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:32.938656 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:32.938610 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:33.037817 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:33.037767 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:33.106244 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:33.106203 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:33.859246 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:33.859217 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:34.069974 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:34.069939 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:00:48.676413 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.676337 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg"] Jul 12 18:00:48.679805 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.679785 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.682180 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.682156 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:00:48.683198 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.683181 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6bjmk\"" Jul 12 18:00:48.683292 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.683217 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:00:48.687262 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.686931 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg"] Jul 12 18:00:48.839453 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.839425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.839666 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.839463 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zgrz\" (UniqueName: \"kubernetes.io/projected/24826af1-295c-41b4-b2e9-de36b2efb604-kube-api-access-2zgrz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.839666 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.839558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.940914 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.940821 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zgrz\" (UniqueName: \"kubernetes.io/projected/24826af1-295c-41b4-b2e9-de36b2efb604-kube-api-access-2zgrz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.941094 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.940921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.941094 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.940983 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.941348 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.941325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.941348 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.941339 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.948762 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.948743 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zgrz\" (UniqueName: \"kubernetes.io/projected/24826af1-295c-41b4-b2e9-de36b2efb604-kube-api-access-2zgrz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:48.990262 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:48.990229 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:49.127319 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.127267 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg"] Jul 12 18:00:49.470910 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.470875 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb"] Jul 12 18:00:49.474231 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.474215 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.482295 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.482264 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb"] Jul 12 18:00:49.646607 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.646553 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.646801 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.646641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.646801 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.646671 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv94m\" (UniqueName: \"kubernetes.io/projected/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-kube-api-access-tv94m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.687435 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.687397 2568 generic.go:356] "Generic (PLEG): container finished" podID="24826af1-295c-41b4-b2e9-de36b2efb604" containerID="24ff496c4feadbac5603e01d067ac3c9f2eb207b87ab329f99344527869ed425" exitCode=0 Jul 12 18:00:49.687889 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.687439 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" event={"ID":"24826af1-295c-41b4-b2e9-de36b2efb604","Type":"ContainerDied","Data":"24ff496c4feadbac5603e01d067ac3c9f2eb207b87ab329f99344527869ed425"} Jul 12 18:00:49.687889 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.687489 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" event={"ID":"24826af1-295c-41b4-b2e9-de36b2efb604","Type":"ContainerStarted","Data":"a4f46d51bf95bec9d19dd997cb92d536a3c48aade04511930d839ac91cd87974"} Jul 12 18:00:49.747991 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.747893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.747991 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.747987 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.748186 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.748026 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv94m\" (UniqueName: \"kubernetes.io/projected/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-kube-api-access-tv94m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.748360 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.748337 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.748411 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.748344 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.756006 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.755982 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv94m\" (UniqueName: \"kubernetes.io/projected/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-kube-api-access-tv94m\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.784606 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.784572 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:49.909718 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:49.909692 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb"] Jul 12 18:00:49.912102 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:00:49.912072 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15ab6d0_1584_46ec_ba13_9c85ca2aa9fe.slice/crio-1db0298f61db8de7e9720dd78fb26024df78e15e23e51defe3efe4e5ec4703d0 WatchSource:0}: Error finding container 1db0298f61db8de7e9720dd78fb26024df78e15e23e51defe3efe4e5ec4703d0: Status 404 returned error can't find the container with id 1db0298f61db8de7e9720dd78fb26024df78e15e23e51defe3efe4e5ec4703d0 Jul 12 18:00:50.077369 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.077332 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh"] Jul 12 18:00:50.081001 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.080976 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.086615 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.086586 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh"] Jul 12 18:00:50.252160 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.252128 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.252316 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.252187 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.252316 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.252242 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x8cb\" (UniqueName: \"kubernetes.io/projected/f7fecf65-edda-4a2c-8050-79823a80ca2b-kube-api-access-2x8cb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.353319 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.353235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.353477 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.353320 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.353477 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.353367 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x8cb\" (UniqueName: \"kubernetes.io/projected/f7fecf65-edda-4a2c-8050-79823a80ca2b-kube-api-access-2x8cb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.353616 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.353600 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.353668 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.353636 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.361398 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.361362 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x8cb\" (UniqueName: \"kubernetes.io/projected/f7fecf65-edda-4a2c-8050-79823a80ca2b-kube-api-access-2x8cb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.397264 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.397233 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:50.476220 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.476195 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t"] Jul 12 18:00:50.480668 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.480648 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.487474 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.487232 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t"] Jul 12 18:00:50.539639 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.539615 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh"] Jul 12 18:00:50.578638 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:00:50.578601 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7fecf65_edda_4a2c_8050_79823a80ca2b.slice/crio-95fe1ea66dde71996694659158330a8f96ee61926e0fa712d6f97db5103cf25c WatchSource:0}: Error finding container 95fe1ea66dde71996694659158330a8f96ee61926e0fa712d6f97db5103cf25c: Status 404 returned error can't find the container with id 95fe1ea66dde71996694659158330a8f96ee61926e0fa712d6f97db5103cf25c Jul 12 18:00:50.656395 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.656357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.656589 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.656443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crh7q\" (UniqueName: \"kubernetes.io/projected/07236e8e-d972-4846-97db-67643cc9a723-kube-api-access-crh7q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.656589 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.656522 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.696112 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.696080 2568 generic.go:356] "Generic (PLEG): container finished" podID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerID="ed9f62986d9c8efef2cb8f841b60b23cb0e35b2d8114e429d57c67cd5a220428" exitCode=0 Jul 12 18:00:50.696542 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.696158 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" event={"ID":"f7fecf65-edda-4a2c-8050-79823a80ca2b","Type":"ContainerDied","Data":"ed9f62986d9c8efef2cb8f841b60b23cb0e35b2d8114e429d57c67cd5a220428"} Jul 12 18:00:50.696542 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.696210 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" event={"ID":"f7fecf65-edda-4a2c-8050-79823a80ca2b","Type":"ContainerStarted","Data":"95fe1ea66dde71996694659158330a8f96ee61926e0fa712d6f97db5103cf25c"} Jul 12 18:00:50.697856 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.697831 2568 generic.go:356] "Generic (PLEG): container finished" podID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerID="835ee477ccabb22a41c7f6359a1ceaecc388395a5c58cf22ef05fac8eec9671b" exitCode=0 Jul 12 18:00:50.697977 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.697889 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" event={"ID":"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe","Type":"ContainerDied","Data":"835ee477ccabb22a41c7f6359a1ceaecc388395a5c58cf22ef05fac8eec9671b"} Jul 12 18:00:50.697977 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.697951 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" event={"ID":"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe","Type":"ContainerStarted","Data":"1db0298f61db8de7e9720dd78fb26024df78e15e23e51defe3efe4e5ec4703d0"} Jul 12 18:00:50.700200 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.700176 2568 generic.go:356] "Generic (PLEG): container finished" podID="24826af1-295c-41b4-b2e9-de36b2efb604" containerID="f944fcc8118dff7bd99adfcc629badcf3b5c854d6e91ef94a12b25ac4c901109" exitCode=0 Jul 12 18:00:50.700305 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.700210 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" event={"ID":"24826af1-295c-41b4-b2e9-de36b2efb604","Type":"ContainerDied","Data":"f944fcc8118dff7bd99adfcc629badcf3b5c854d6e91ef94a12b25ac4c901109"} Jul 12 18:00:50.757903 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.757877 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.758050 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.757914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.758050 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.757955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crh7q\" (UniqueName: \"kubernetes.io/projected/07236e8e-d972-4846-97db-67643cc9a723-kube-api-access-crh7q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.758256 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.758237 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.758322 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.758304 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.766163 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.766141 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crh7q\" (UniqueName: \"kubernetes.io/projected/07236e8e-d972-4846-97db-67643cc9a723-kube-api-access-crh7q\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.795107 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.795078 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:50.929422 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:50.929396 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t"] Jul 12 18:00:50.931556 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:00:50.931522 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07236e8e_d972_4846_97db_67643cc9a723.slice/crio-b68bbf59e8ca2375132573fd1376089cf02c912c5a8216caf44f304a994a4f81 WatchSource:0}: Error finding container b68bbf59e8ca2375132573fd1376089cf02c912c5a8216caf44f304a994a4f81: Status 404 returned error can't find the container with id b68bbf59e8ca2375132573fd1376089cf02c912c5a8216caf44f304a994a4f81 Jul 12 18:00:51.708142 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.708110 2568 generic.go:356] "Generic (PLEG): container finished" podID="07236e8e-d972-4846-97db-67643cc9a723" containerID="f2e4779076c6d96042165fd4720eecfbebdc0e78f2d9ae32bb04490fcda210cc" exitCode=0 Jul 12 18:00:51.708545 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.708203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" event={"ID":"07236e8e-d972-4846-97db-67643cc9a723","Type":"ContainerDied","Data":"f2e4779076c6d96042165fd4720eecfbebdc0e78f2d9ae32bb04490fcda210cc"} Jul 12 18:00:51.708545 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.708293 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" event={"ID":"07236e8e-d972-4846-97db-67643cc9a723","Type":"ContainerStarted","Data":"b68bbf59e8ca2375132573fd1376089cf02c912c5a8216caf44f304a994a4f81"} Jul 12 18:00:51.710037 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.710015 2568 generic.go:356] "Generic (PLEG): container finished" podID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerID="19d1b56e3572a5fef80e2492faac6ee1b0905c510c8b1edf3b585ff8b757359d" exitCode=0 Jul 12 18:00:51.710161 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.710095 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" event={"ID":"f7fecf65-edda-4a2c-8050-79823a80ca2b","Type":"ContainerDied","Data":"19d1b56e3572a5fef80e2492faac6ee1b0905c510c8b1edf3b585ff8b757359d"} Jul 12 18:00:51.712031 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.712013 2568 generic.go:356] "Generic (PLEG): container finished" podID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerID="e693a0715b16e58d8a7f4b6ffa81042d7adf7bdb92ae0c70fa64089c489f01af" exitCode=0 Jul 12 18:00:51.712144 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.712102 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" event={"ID":"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe","Type":"ContainerDied","Data":"e693a0715b16e58d8a7f4b6ffa81042d7adf7bdb92ae0c70fa64089c489f01af"} Jul 12 18:00:51.714391 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.714371 2568 generic.go:356] "Generic (PLEG): container finished" podID="24826af1-295c-41b4-b2e9-de36b2efb604" containerID="d04995b786700aa4a83040c07273915ab6fda98498682e3274ee419246348e2a" exitCode=0 Jul 12 18:00:51.714484 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:51.714401 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" event={"ID":"24826af1-295c-41b4-b2e9-de36b2efb604","Type":"ContainerDied","Data":"d04995b786700aa4a83040c07273915ab6fda98498682e3274ee419246348e2a"} Jul 12 18:00:52.720768 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:52.720732 2568 generic.go:356] "Generic (PLEG): container finished" podID="07236e8e-d972-4846-97db-67643cc9a723" containerID="fb5e20ffc909dc0324a89b4cd067364bb7079db8fe06d6cf33259cbe32d58b7e" exitCode=0 Jul 12 18:00:52.721223 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:52.720813 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" event={"ID":"07236e8e-d972-4846-97db-67643cc9a723","Type":"ContainerDied","Data":"fb5e20ffc909dc0324a89b4cd067364bb7079db8fe06d6cf33259cbe32d58b7e"} Jul 12 18:00:52.723003 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:52.722982 2568 generic.go:356] "Generic (PLEG): container finished" podID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerID="2fda14a8d65cf71ddbfb68227267c6e3d76579d5b1bbeed4a1e80cd28f3d8509" exitCode=0 Jul 12 18:00:52.723084 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:52.723056 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" event={"ID":"f7fecf65-edda-4a2c-8050-79823a80ca2b","Type":"ContainerDied","Data":"2fda14a8d65cf71ddbfb68227267c6e3d76579d5b1bbeed4a1e80cd28f3d8509"} Jul 12 18:00:52.724909 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:52.724893 2568 generic.go:356] "Generic (PLEG): container finished" podID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerID="0e8867e6e53aed2762efbb4cb30bd24ff328ab0e07a89d12e7b084109ac1a2ed" exitCode=0 Jul 12 18:00:52.724989 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:52.724949 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" event={"ID":"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe","Type":"ContainerDied","Data":"0e8867e6e53aed2762efbb4cb30bd24ff328ab0e07a89d12e7b084109ac1a2ed"} Jul 12 18:00:52.920561 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:52.920535 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:53.077320 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.077274 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-util\") pod \"24826af1-295c-41b4-b2e9-de36b2efb604\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " Jul 12 18:00:53.077536 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.077358 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-bundle\") pod \"24826af1-295c-41b4-b2e9-de36b2efb604\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " Jul 12 18:00:53.077536 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.077405 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zgrz\" (UniqueName: \"kubernetes.io/projected/24826af1-295c-41b4-b2e9-de36b2efb604-kube-api-access-2zgrz\") pod \"24826af1-295c-41b4-b2e9-de36b2efb604\" (UID: \"24826af1-295c-41b4-b2e9-de36b2efb604\") " Jul 12 18:00:53.077832 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.077806 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-bundle" (OuterVolumeSpecName: "bundle") pod "24826af1-295c-41b4-b2e9-de36b2efb604" (UID: "24826af1-295c-41b4-b2e9-de36b2efb604"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:00:53.079615 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.079595 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24826af1-295c-41b4-b2e9-de36b2efb604-kube-api-access-2zgrz" (OuterVolumeSpecName: "kube-api-access-2zgrz") pod "24826af1-295c-41b4-b2e9-de36b2efb604" (UID: "24826af1-295c-41b4-b2e9-de36b2efb604"). InnerVolumeSpecName "kube-api-access-2zgrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:00:53.085009 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.084986 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-util" (OuterVolumeSpecName: "util") pod "24826af1-295c-41b4-b2e9-de36b2efb604" (UID: "24826af1-295c-41b4-b2e9-de36b2efb604"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:00:53.178600 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.178567 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:53.178600 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.178594 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zgrz\" (UniqueName: \"kubernetes.io/projected/24826af1-295c-41b4-b2e9-de36b2efb604-kube-api-access-2zgrz\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:53.178600 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.178603 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24826af1-295c-41b4-b2e9-de36b2efb604-util\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:53.731467 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.731435 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" Jul 12 18:00:53.732059 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.731431 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg" event={"ID":"24826af1-295c-41b4-b2e9-de36b2efb604","Type":"ContainerDied","Data":"a4f46d51bf95bec9d19dd997cb92d536a3c48aade04511930d839ac91cd87974"} Jul 12 18:00:53.732059 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.731574 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4f46d51bf95bec9d19dd997cb92d536a3c48aade04511930d839ac91cd87974" Jul 12 18:00:53.733603 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.733583 2568 generic.go:356] "Generic (PLEG): container finished" podID="07236e8e-d972-4846-97db-67643cc9a723" containerID="3a273664b43a9508a29f1bd867455bc75b982e2146c9ee55ff925f9422763a36" exitCode=0 Jul 12 18:00:53.733704 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.733650 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" event={"ID":"07236e8e-d972-4846-97db-67643cc9a723","Type":"ContainerDied","Data":"3a273664b43a9508a29f1bd867455bc75b982e2146c9ee55ff925f9422763a36"} Jul 12 18:00:53.863685 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.863656 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:53.886279 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.886258 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:53.986320 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.986245 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-bundle\") pod \"f7fecf65-edda-4a2c-8050-79823a80ca2b\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " Jul 12 18:00:53.986448 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.986333 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x8cb\" (UniqueName: \"kubernetes.io/projected/f7fecf65-edda-4a2c-8050-79823a80ca2b-kube-api-access-2x8cb\") pod \"f7fecf65-edda-4a2c-8050-79823a80ca2b\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " Jul 12 18:00:53.986448 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.986371 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-bundle\") pod \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " Jul 12 18:00:53.986448 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.986416 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-util\") pod \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " Jul 12 18:00:53.986626 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.986454 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv94m\" (UniqueName: \"kubernetes.io/projected/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-kube-api-access-tv94m\") pod \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\" (UID: \"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe\") " Jul 12 18:00:53.986626 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.986481 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-util\") pod \"f7fecf65-edda-4a2c-8050-79823a80ca2b\" (UID: \"f7fecf65-edda-4a2c-8050-79823a80ca2b\") " Jul 12 18:00:53.986966 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.986936 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-bundle" (OuterVolumeSpecName: "bundle") pod "f7fecf65-edda-4a2c-8050-79823a80ca2b" (UID: "f7fecf65-edda-4a2c-8050-79823a80ca2b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:00:53.987148 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.987120 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-bundle" (OuterVolumeSpecName: "bundle") pod "b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" (UID: "b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:00:53.988584 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.988556 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-kube-api-access-tv94m" (OuterVolumeSpecName: "kube-api-access-tv94m") pod "b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" (UID: "b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe"). InnerVolumeSpecName "kube-api-access-tv94m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:00:53.988827 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.988805 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fecf65-edda-4a2c-8050-79823a80ca2b-kube-api-access-2x8cb" (OuterVolumeSpecName: "kube-api-access-2x8cb") pod "f7fecf65-edda-4a2c-8050-79823a80ca2b" (UID: "f7fecf65-edda-4a2c-8050-79823a80ca2b"). InnerVolumeSpecName "kube-api-access-2x8cb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:00:53.992467 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.992444 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-util" (OuterVolumeSpecName: "util") pod "b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" (UID: "b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:00:53.992797 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:53.992738 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-util" (OuterVolumeSpecName: "util") pod "f7fecf65-edda-4a2c-8050-79823a80ca2b" (UID: "f7fecf65-edda-4a2c-8050-79823a80ca2b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:00:54.087389 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.087350 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x8cb\" (UniqueName: \"kubernetes.io/projected/f7fecf65-edda-4a2c-8050-79823a80ca2b-kube-api-access-2x8cb\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:54.087389 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.087383 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:54.087389 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.087392 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-util\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:54.087389 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.087401 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv94m\" (UniqueName: \"kubernetes.io/projected/b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe-kube-api-access-tv94m\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:54.087688 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.087410 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-util\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:54.087688 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.087418 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7fecf65-edda-4a2c-8050-79823a80ca2b-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:54.740957 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.740929 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" Jul 12 18:00:54.741420 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.740956 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb" event={"ID":"b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe","Type":"ContainerDied","Data":"1db0298f61db8de7e9720dd78fb26024df78e15e23e51defe3efe4e5ec4703d0"} Jul 12 18:00:54.741420 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.741046 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1db0298f61db8de7e9720dd78fb26024df78e15e23e51defe3efe4e5ec4703d0" Jul 12 18:00:54.743009 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.742971 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" event={"ID":"f7fecf65-edda-4a2c-8050-79823a80ca2b","Type":"ContainerDied","Data":"95fe1ea66dde71996694659158330a8f96ee61926e0fa712d6f97db5103cf25c"} Jul 12 18:00:54.743139 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.743017 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95fe1ea66dde71996694659158330a8f96ee61926e0fa712d6f97db5103cf25c" Jul 12 18:00:54.743212 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.743195 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh" Jul 12 18:00:54.880133 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.880108 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:54.995713 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.995634 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-bundle\") pod \"07236e8e-d972-4846-97db-67643cc9a723\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " Jul 12 18:00:54.995713 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.995685 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crh7q\" (UniqueName: \"kubernetes.io/projected/07236e8e-d972-4846-97db-67643cc9a723-kube-api-access-crh7q\") pod \"07236e8e-d972-4846-97db-67643cc9a723\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " Jul 12 18:00:54.995915 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.995718 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-util\") pod \"07236e8e-d972-4846-97db-67643cc9a723\" (UID: \"07236e8e-d972-4846-97db-67643cc9a723\") " Jul 12 18:00:54.996246 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.996216 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-bundle" (OuterVolumeSpecName: "bundle") pod "07236e8e-d972-4846-97db-67643cc9a723" (UID: "07236e8e-d972-4846-97db-67643cc9a723"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:00:54.997821 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:54.997792 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07236e8e-d972-4846-97db-67643cc9a723-kube-api-access-crh7q" (OuterVolumeSpecName: "kube-api-access-crh7q") pod "07236e8e-d972-4846-97db-67643cc9a723" (UID: "07236e8e-d972-4846-97db-67643cc9a723"). InnerVolumeSpecName "kube-api-access-crh7q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:00:55.003518 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:55.003468 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-util" (OuterVolumeSpecName: "util") pod "07236e8e-d972-4846-97db-67643cc9a723" (UID: "07236e8e-d972-4846-97db-67643cc9a723"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:00:55.096166 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:55.096135 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:55.096166 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:55.096162 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crh7q\" (UniqueName: \"kubernetes.io/projected/07236e8e-d972-4846-97db-67643cc9a723-kube-api-access-crh7q\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:55.096166 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:55.096172 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07236e8e-d972-4846-97db-67643cc9a723-util\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:00:55.749324 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:55.749291 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" event={"ID":"07236e8e-d972-4846-97db-67643cc9a723","Type":"ContainerDied","Data":"b68bbf59e8ca2375132573fd1376089cf02c912c5a8216caf44f304a994a4f81"} Jul 12 18:00:55.749732 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:55.749334 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t" Jul 12 18:00:55.749732 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:00:55.749336 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b68bbf59e8ca2375132573fd1376089cf02c912c5a8216caf44f304a994a4f81" Jul 12 18:01:02.478183 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:02.478148 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:04.096902 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:04.096867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:06.675978 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:06.675950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:09.078726 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:09.078689 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:09.319844 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:09.319812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:09.687090 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:09.687056 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:12.339983 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:12.339934 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:13.071923 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:13.071873 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:14.154923 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:14.154894 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:14.250798 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:14.250751 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:16.121873 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:16.121837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:17.927368 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:17.927332 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:19.293986 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:19.293939 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:19.395249 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:19.395206 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:23.080087 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:23.080057 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:23.183295 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:23.183258 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:23.654433 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:23.654396 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff8f4d8cd-wn7zk"] Jul 12 18:01:23.751876 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:23.751842 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:23.908968 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:23.908884 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:23.988373 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:23.988343 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:26.068011 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:26.067982 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:26.497227 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:26.497193 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:26.596394 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:26.596366 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:26.697475 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:26.697445 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:27.279275 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:27.279244 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:27.343810 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:27.343779 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:27.673819 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:27.673788 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:27.729578 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:27.729548 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:31.448313 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:31.448276 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:31.548381 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:31.548348 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:32.791714 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:32.791679 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:36.223234 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223205 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf"] Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223493 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerName="util" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223519 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerName="util" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223531 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24826af1-295c-41b4-b2e9-de36b2efb604" containerName="util" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223536 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="24826af1-295c-41b4-b2e9-de36b2efb604" containerName="util" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223542 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24826af1-295c-41b4-b2e9-de36b2efb604" containerName="extract" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223547 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="24826af1-295c-41b4-b2e9-de36b2efb604" containerName="extract" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223553 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerName="pull" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223558 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerName="pull" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223566 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerName="util" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223571 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerName="util" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223577 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerName="extract" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223582 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerName="extract" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223588 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24826af1-295c-41b4-b2e9-de36b2efb604" containerName="pull" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223593 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="24826af1-295c-41b4-b2e9-de36b2efb604" containerName="pull" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223599 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerName="pull" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223604 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerName="pull" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223610 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07236e8e-d972-4846-97db-67643cc9a723" containerName="extract" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223614 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="07236e8e-d972-4846-97db-67643cc9a723" containerName="extract" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223624 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07236e8e-d972-4846-97db-67643cc9a723" containerName="pull" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223630 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="07236e8e-d972-4846-97db-67643cc9a723" containerName="pull" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223637 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07236e8e-d972-4846-97db-67643cc9a723" containerName="util" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223642 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="07236e8e-d972-4846-97db-67643cc9a723" containerName="util" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223648 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerName="extract" Jul 12 18:01:36.223660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223653 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerName="extract" Jul 12 18:01:36.224454 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223697 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fecf65-edda-4a2c-8050-79823a80ca2b" containerName="extract" Jul 12 18:01:36.224454 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223704 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe" containerName="extract" Jul 12 18:01:36.224454 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223711 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="24826af1-295c-41b4-b2e9-de36b2efb604" containerName="extract" Jul 12 18:01:36.224454 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.223716 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="07236e8e-d972-4846-97db-67643cc9a723" containerName="extract" Jul 12 18:01:36.227877 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.227859 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:36.230668 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.230650 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:01:36.230776 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.230665 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:01:36.230776 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.230670 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tjgw4\"" Jul 12 18:01:36.238171 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.238149 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf"] Jul 12 18:01:36.326523 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.326479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnbsw\" (UniqueName: \"kubernetes.io/projected/bd222326-ebb0-450f-b3c7-b421bcbb0443-kube-api-access-bnbsw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lcpdf\" (UID: \"bd222326-ebb0-450f-b3c7-b421bcbb0443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:36.326667 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.326616 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd222326-ebb0-450f-b3c7-b421bcbb0443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lcpdf\" (UID: \"bd222326-ebb0-450f-b3c7-b421bcbb0443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:36.427856 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.427810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd222326-ebb0-450f-b3c7-b421bcbb0443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lcpdf\" (UID: \"bd222326-ebb0-450f-b3c7-b421bcbb0443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:36.428033 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.427874 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnbsw\" (UniqueName: \"kubernetes.io/projected/bd222326-ebb0-450f-b3c7-b421bcbb0443-kube-api-access-bnbsw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lcpdf\" (UID: \"bd222326-ebb0-450f-b3c7-b421bcbb0443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:36.428202 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.428182 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd222326-ebb0-450f-b3c7-b421bcbb0443-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lcpdf\" (UID: \"bd222326-ebb0-450f-b3c7-b421bcbb0443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:36.436282 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.436265 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnbsw\" (UniqueName: \"kubernetes.io/projected/bd222326-ebb0-450f-b3c7-b421bcbb0443-kube-api-access-bnbsw\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lcpdf\" (UID: \"bd222326-ebb0-450f-b3c7-b421bcbb0443\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:36.539448 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.539369 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:36.671725 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.671699 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf"] Jul 12 18:01:36.673547 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:01:36.673522 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd222326_ebb0_450f_b3c7_b421bcbb0443.slice/crio-feef0fe295adc96dce80b65b17803cd7841c99cd2395d4f47d62941b8796eeea WatchSource:0}: Error finding container feef0fe295adc96dce80b65b17803cd7841c99cd2395d4f47d62941b8796eeea: Status 404 returned error can't find the container with id feef0fe295adc96dce80b65b17803cd7841c99cd2395d4f47d62941b8796eeea Jul 12 18:01:36.937088 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.937062 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" event={"ID":"bd222326-ebb0-450f-b3c7-b421bcbb0443","Type":"ContainerStarted","Data":"feef0fe295adc96dce80b65b17803cd7841c99cd2395d4f47d62941b8796eeea"} Jul 12 18:01:36.949113 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.949074 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf"] Jul 12 18:01:36.953325 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.953305 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf"] Jul 12 18:01:36.972984 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.972960 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7"] Jul 12 18:01:36.977439 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.977420 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:36.993669 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:36.993642 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7"] Jul 12 18:01:37.031841 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.031812 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/581edc88-282a-4d9d-9d5f-fae933222059-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gbzb7\" (UID: \"581edc88-282a-4d9d-9d5f-fae933222059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:37.031841 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.031843 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvp57\" (UniqueName: \"kubernetes.io/projected/581edc88-282a-4d9d-9d5f-fae933222059-kube-api-access-nvp57\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gbzb7\" (UID: \"581edc88-282a-4d9d-9d5f-fae933222059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:37.132272 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.132228 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/581edc88-282a-4d9d-9d5f-fae933222059-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gbzb7\" (UID: \"581edc88-282a-4d9d-9d5f-fae933222059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:37.132272 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.132264 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvp57\" (UniqueName: \"kubernetes.io/projected/581edc88-282a-4d9d-9d5f-fae933222059-kube-api-access-nvp57\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gbzb7\" (UID: \"581edc88-282a-4d9d-9d5f-fae933222059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:37.132636 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.132614 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/581edc88-282a-4d9d-9d5f-fae933222059-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gbzb7\" (UID: \"581edc88-282a-4d9d-9d5f-fae933222059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:37.151932 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.151905 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvp57\" (UniqueName: \"kubernetes.io/projected/581edc88-282a-4d9d-9d5f-fae933222059-kube-api-access-nvp57\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gbzb7\" (UID: \"581edc88-282a-4d9d-9d5f-fae933222059\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:37.290131 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.290060 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:37.460810 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.460789 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7"] Jul 12 18:01:37.479434 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:01:37.479407 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581edc88_282a_4d9d_9d5f_fae933222059.slice/crio-42e08d07954e6bfbd8311a560ab72cd4d09e1ba6fc499381417c85f28c09b1fe WatchSource:0}: Error finding container 42e08d07954e6bfbd8311a560ab72cd4d09e1ba6fc499381417c85f28c09b1fe: Status 404 returned error can't find the container with id 42e08d07954e6bfbd8311a560ab72cd4d09e1ba6fc499381417c85f28c09b1fe Jul 12 18:01:37.945729 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:37.945682 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" event={"ID":"581edc88-282a-4d9d-9d5f-fae933222059","Type":"ContainerStarted","Data":"42e08d07954e6bfbd8311a560ab72cd4d09e1ba6fc499381417c85f28c09b1fe"} Jul 12 18:01:38.104198 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:38.104138 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:38.208650 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:38.208461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:40.944530 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:40.944481 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:41.044233 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:41.044174 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:41.969119 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:41.969053 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" podUID="bd222326-ebb0-450f-b3c7-b421bcbb0443" containerName="manager" containerID="cri-o://4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df" gracePeriod=2 Jul 12 18:01:41.970522 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:41.970481 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" event={"ID":"581edc88-282a-4d9d-9d5f-fae933222059","Type":"ContainerStarted","Data":"b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983"} Jul 12 18:01:41.970711 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:41.970689 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:01:41.990841 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:41.990797 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" podStartSLOduration=2.51344306 podStartE2EDuration="5.990782883s" totalImagesPullingTime="3.477339823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:01:41.988079135 +0000 UTC m=+495.631921895" watchObservedRunningTime="2026-07-12 18:01:41.990782883 +0000 UTC m=+495.634625606" Jul 12 18:01:42.198569 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.198546 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:42.201004 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:01:42.200977 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lcpdf\" is forbidden: User \"system:node:ip-10-0-143-136.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-136.ec2.internal' and this object" podUID="bd222326-ebb0-450f-b3c7-b421bcbb0443" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:42.279181 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.279104 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd222326-ebb0-450f-b3c7-b421bcbb0443-extensions-socket-volume\") pod \"bd222326-ebb0-450f-b3c7-b421bcbb0443\" (UID: \"bd222326-ebb0-450f-b3c7-b421bcbb0443\") " Jul 12 18:01:42.279311 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.279187 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnbsw\" (UniqueName: \"kubernetes.io/projected/bd222326-ebb0-450f-b3c7-b421bcbb0443-kube-api-access-bnbsw\") pod \"bd222326-ebb0-450f-b3c7-b421bcbb0443\" (UID: \"bd222326-ebb0-450f-b3c7-b421bcbb0443\") " Jul 12 18:01:42.279413 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.279389 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd222326-ebb0-450f-b3c7-b421bcbb0443-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bd222326-ebb0-450f-b3c7-b421bcbb0443" (UID: "bd222326-ebb0-450f-b3c7-b421bcbb0443"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:01:42.281135 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.281113 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd222326-ebb0-450f-b3c7-b421bcbb0443-kube-api-access-bnbsw" (OuterVolumeSpecName: "kube-api-access-bnbsw") pod "bd222326-ebb0-450f-b3c7-b421bcbb0443" (UID: "bd222326-ebb0-450f-b3c7-b421bcbb0443"). InnerVolumeSpecName "kube-api-access-bnbsw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:01:42.380700 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.380671 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnbsw\" (UniqueName: \"kubernetes.io/projected/bd222326-ebb0-450f-b3c7-b421bcbb0443-kube-api-access-bnbsw\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:42.380700 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.380696 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd222326-ebb0-450f-b3c7-b421bcbb0443-extensions-socket-volume\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:42.834729 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.834703 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd222326-ebb0-450f-b3c7-b421bcbb0443" path="/var/lib/kubelet/pods/bd222326-ebb0-450f-b3c7-b421bcbb0443/volumes" Jul 12 18:01:42.975761 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.975730 2568 generic.go:356] "Generic (PLEG): container finished" podID="bd222326-ebb0-450f-b3c7-b421bcbb0443" containerID="4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df" exitCode=2 Jul 12 18:01:42.976201 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.975772 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:42.976201 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.975819 2568 scope.go:117] "RemoveContainer" containerID="4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df" Jul 12 18:01:42.980116 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:01:42.980090 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lcpdf\" is forbidden: User \"system:node:ip-10-0-143-136.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-136.ec2.internal' and this object" podUID="bd222326-ebb0-450f-b3c7-b421bcbb0443" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lcpdf" Jul 12 18:01:42.984160 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.984142 2568 scope.go:117] "RemoveContainer" containerID="4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df" Jul 12 18:01:42.984405 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:01:42.984389 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df\": container with ID starting with 4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df not found: ID does not exist" containerID="4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df" Jul 12 18:01:42.984453 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:42.984414 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df"} err="failed to get container status \"4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df\": rpc error: code = NotFound desc = could not find container \"4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df\": container with ID starting with 4911bcff88b709fe93cf4c3c836d8a76c5700cf2c0b73f44487883838b86e2df not found: ID does not exist" Jul 12 18:01:45.726071 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:45.726034 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:45.823353 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:45.823321 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:48.676575 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:48.676527 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6ff8f4d8cd-wn7zk" podUID="2756b9c0-3752-4508-970f-16f5c66308fe" containerName="console" containerID="cri-o://99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d" gracePeriod=15 Jul 12 18:01:48.921452 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:48.921424 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 18:01:49.005998 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.005910 2568 generic.go:356] "Generic (PLEG): container finished" podID="2756b9c0-3752-4508-970f-16f5c66308fe" containerID="99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d" exitCode=0 Jul 12 18:01:49.005998 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.005955 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ff8f4d8cd-wn7zk" Jul 12 18:01:49.006214 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.005993 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff8f4d8cd-wn7zk" event={"ID":"2756b9c0-3752-4508-970f-16f5c66308fe","Type":"ContainerDied","Data":"99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d"} Jul 12 18:01:49.006214 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.006046 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ff8f4d8cd-wn7zk" event={"ID":"2756b9c0-3752-4508-970f-16f5c66308fe","Type":"ContainerDied","Data":"ac5310c58fe7e8cfb9b0494c5b9ea8d5a013906d98bb174e9645d6631c992761"} Jul 12 18:01:49.006214 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.006073 2568 scope.go:117] "RemoveContainer" containerID="99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d" Jul 12 18:01:49.014009 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.013999 2568 scope.go:117] "RemoveContainer" containerID="99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d" Jul 12 18:01:49.014242 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:01:49.014226 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d\": container with ID starting with 99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d not found: ID does not exist" containerID="99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d" Jul 12 18:01:49.014292 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.014253 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d"} err="failed to get container status \"99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d\": rpc error: code = NotFound desc = could not find container \"99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d\": container with ID starting with 99c356c4f5d7e9555f063bf434017dbb4d942059b6f764bd25da1f0aa6770b9d not found: ID does not exist" Jul 12 18:01:49.039805 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.039775 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-serving-cert\") pod \"2756b9c0-3752-4508-970f-16f5c66308fe\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " Jul 12 18:01:49.039944 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.039817 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-service-ca\") pod \"2756b9c0-3752-4508-970f-16f5c66308fe\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " Jul 12 18:01:49.039944 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.039851 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-oauth-serving-cert\") pod \"2756b9c0-3752-4508-970f-16f5c66308fe\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " Jul 12 18:01:49.039944 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.039873 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-oauth-config\") pod \"2756b9c0-3752-4508-970f-16f5c66308fe\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " Jul 12 18:01:49.040101 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.039960 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-console-config\") pod \"2756b9c0-3752-4508-970f-16f5c66308fe\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " Jul 12 18:01:49.040101 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.039985 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qv8v\" (UniqueName: \"kubernetes.io/projected/2756b9c0-3752-4508-970f-16f5c66308fe-kube-api-access-7qv8v\") pod \"2756b9c0-3752-4508-970f-16f5c66308fe\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " Jul 12 18:01:49.040101 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.040022 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-trusted-ca-bundle\") pod \"2756b9c0-3752-4508-970f-16f5c66308fe\" (UID: \"2756b9c0-3752-4508-970f-16f5c66308fe\") " Jul 12 18:01:49.040267 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.040240 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2756b9c0-3752-4508-970f-16f5c66308fe" (UID: "2756b9c0-3752-4508-970f-16f5c66308fe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:01:49.040358 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.040259 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-service-ca" (OuterVolumeSpecName: "service-ca") pod "2756b9c0-3752-4508-970f-16f5c66308fe" (UID: "2756b9c0-3752-4508-970f-16f5c66308fe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:01:49.040358 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.040297 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-console-config" (OuterVolumeSpecName: "console-config") pod "2756b9c0-3752-4508-970f-16f5c66308fe" (UID: "2756b9c0-3752-4508-970f-16f5c66308fe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:01:49.040564 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.040535 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2756b9c0-3752-4508-970f-16f5c66308fe" (UID: "2756b9c0-3752-4508-970f-16f5c66308fe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:01:49.042093 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.042068 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2756b9c0-3752-4508-970f-16f5c66308fe" (UID: "2756b9c0-3752-4508-970f-16f5c66308fe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:01:49.042183 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.042120 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2756b9c0-3752-4508-970f-16f5c66308fe" (UID: "2756b9c0-3752-4508-970f-16f5c66308fe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:01:49.042183 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.042133 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2756b9c0-3752-4508-970f-16f5c66308fe-kube-api-access-7qv8v" (OuterVolumeSpecName: "kube-api-access-7qv8v") pod "2756b9c0-3752-4508-970f-16f5c66308fe" (UID: "2756b9c0-3752-4508-970f-16f5c66308fe"). InnerVolumeSpecName "kube-api-access-7qv8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:01:49.140727 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.140690 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-console-config\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:49.140727 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.140720 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qv8v\" (UniqueName: \"kubernetes.io/projected/2756b9c0-3752-4508-970f-16f5c66308fe-kube-api-access-7qv8v\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:49.140727 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.140731 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-trusted-ca-bundle\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:49.140956 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.140739 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-serving-cert\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:49.140956 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.140748 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:49.140956 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.140756 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2756b9c0-3752-4508-970f-16f5c66308fe-oauth-serving-cert\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:49.140956 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.140763 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2756b9c0-3752-4508-970f-16f5c66308fe-console-oauth-config\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:01:49.265550 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.265459 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:49.328028 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.327997 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ff8f4d8cd-wn7zk"] Jul 12 18:01:49.331772 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.331742 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:01:49.333094 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:49.333072 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6ff8f4d8cd-wn7zk"] Jul 12 18:01:50.835595 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:50.835562 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2756b9c0-3752-4508-970f-16f5c66308fe" path="/var/lib/kubelet/pods/2756b9c0-3752-4508-970f-16f5c66308fe/volumes" Jul 12 18:01:52.978003 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:01:52.977961 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:02:00.538872 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:00.538842 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:00.636724 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:00.636695 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:03.736784 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:03.736752 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:03.838011 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:03.837972 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:05.503190 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.503157 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7"] Jul 12 18:02:05.503561 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.503371 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" podUID="581edc88-282a-4d9d-9d5f-fae933222059" containerName="manager" containerID="cri-o://b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983" gracePeriod=10 Jul 12 18:02:05.748325 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.748300 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:02:05.777135 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.777065 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvp57\" (UniqueName: \"kubernetes.io/projected/581edc88-282a-4d9d-9d5f-fae933222059-kube-api-access-nvp57\") pod \"581edc88-282a-4d9d-9d5f-fae933222059\" (UID: \"581edc88-282a-4d9d-9d5f-fae933222059\") " Jul 12 18:02:05.777135 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.777095 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/581edc88-282a-4d9d-9d5f-fae933222059-extensions-socket-volume\") pod \"581edc88-282a-4d9d-9d5f-fae933222059\" (UID: \"581edc88-282a-4d9d-9d5f-fae933222059\") " Jul 12 18:02:05.777782 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.777751 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581edc88-282a-4d9d-9d5f-fae933222059-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "581edc88-282a-4d9d-9d5f-fae933222059" (UID: "581edc88-282a-4d9d-9d5f-fae933222059"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:02:05.779119 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.779087 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581edc88-282a-4d9d-9d5f-fae933222059-kube-api-access-nvp57" (OuterVolumeSpecName: "kube-api-access-nvp57") pod "581edc88-282a-4d9d-9d5f-fae933222059" (UID: "581edc88-282a-4d9d-9d5f-fae933222059"). InnerVolumeSpecName "kube-api-access-nvp57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:02:05.878425 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.878392 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvp57\" (UniqueName: \"kubernetes.io/projected/581edc88-282a-4d9d-9d5f-fae933222059-kube-api-access-nvp57\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:02:05.878425 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:05.878419 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/581edc88-282a-4d9d-9d5f-fae933222059-extensions-socket-volume\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:02:06.088576 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.088472 2568 generic.go:356] "Generic (PLEG): container finished" podID="581edc88-282a-4d9d-9d5f-fae933222059" containerID="b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983" exitCode=0 Jul 12 18:02:06.088576 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.088533 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" Jul 12 18:02:06.088576 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.088556 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" event={"ID":"581edc88-282a-4d9d-9d5f-fae933222059","Type":"ContainerDied","Data":"b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983"} Jul 12 18:02:06.088802 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.088606 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7" event={"ID":"581edc88-282a-4d9d-9d5f-fae933222059","Type":"ContainerDied","Data":"42e08d07954e6bfbd8311a560ab72cd4d09e1ba6fc499381417c85f28c09b1fe"} Jul 12 18:02:06.088802 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.088629 2568 scope.go:117] "RemoveContainer" containerID="b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983" Jul 12 18:02:06.097252 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.097237 2568 scope.go:117] "RemoveContainer" containerID="b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983" Jul 12 18:02:06.097490 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:02:06.097473 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983\": container with ID starting with b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983 not found: ID does not exist" containerID="b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983" Jul 12 18:02:06.097578 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.097498 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983"} err="failed to get container status \"b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983\": rpc error: code = NotFound desc = could not find container \"b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983\": container with ID starting with b9833d7af23a929aaec328a8857d29ca51c0a024a32c651090621f105d9dc983 not found: ID does not exist" Jul 12 18:02:06.113237 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.113215 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7"] Jul 12 18:02:06.116443 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.116423 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gbzb7"] Jul 12 18:02:06.834156 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:06.834129 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="581edc88-282a-4d9d-9d5f-fae933222059" path="/var/lib/kubelet/pods/581edc88-282a-4d9d-9d5f-fae933222059/volumes" Jul 12 18:02:08.158215 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:08.158184 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:08.258927 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:08.258892 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:09.481025 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:09.480987 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:09.867908 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:09.867828 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:11.691481 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:11.691447 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:11.788798 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:11.788765 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:23.490808 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.490729 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6kgrv"] Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.490993 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd222326-ebb0-450f-b3c7-b421bcbb0443" containerName="manager" Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.491003 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd222326-ebb0-450f-b3c7-b421bcbb0443" containerName="manager" Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.491020 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="581edc88-282a-4d9d-9d5f-fae933222059" containerName="manager" Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.491025 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="581edc88-282a-4d9d-9d5f-fae933222059" containerName="manager" Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.491033 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2756b9c0-3752-4508-970f-16f5c66308fe" containerName="console" Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.491051 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2756b9c0-3752-4508-970f-16f5c66308fe" containerName="console" Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.491101 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd222326-ebb0-450f-b3c7-b421bcbb0443" containerName="manager" Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.491108 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="581edc88-282a-4d9d-9d5f-fae933222059" containerName="manager" Jul 12 18:02:23.491168 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.491114 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2756b9c0-3752-4508-970f-16f5c66308fe" containerName="console" Jul 12 18:02:23.494018 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.493993 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:23.494132 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.494001 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:23.496322 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.496303 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 12 18:02:23.496438 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.496349 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:02:23.496438 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.496407 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:02:23.497290 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.497272 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6bjmk\"" Jul 12 18:02:23.500685 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.500664 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6kgrv"] Jul 12 18:02:23.584176 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.584142 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6kgrv"] Jul 12 18:02:23.595795 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.595734 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:23.618228 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.618205 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bb59438-f2b5-41f4-b963-03afaac575c7-config-file\") pod \"limitador-limitador-64c94fd64f-6kgrv\" (UID: \"4bb59438-f2b5-41f4-b963-03afaac575c7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:23.618334 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.618241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7mqk\" (UniqueName: \"kubernetes.io/projected/4bb59438-f2b5-41f4-b963-03afaac575c7-kube-api-access-w7mqk\") pod \"limitador-limitador-64c94fd64f-6kgrv\" (UID: \"4bb59438-f2b5-41f4-b963-03afaac575c7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:23.649568 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.649545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:23.718943 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.718917 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7mqk\" (UniqueName: \"kubernetes.io/projected/4bb59438-f2b5-41f4-b963-03afaac575c7-kube-api-access-w7mqk\") pod \"limitador-limitador-64c94fd64f-6kgrv\" (UID: \"4bb59438-f2b5-41f4-b963-03afaac575c7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:23.719066 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.718980 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bb59438-f2b5-41f4-b963-03afaac575c7-config-file\") pod \"limitador-limitador-64c94fd64f-6kgrv\" (UID: \"4bb59438-f2b5-41f4-b963-03afaac575c7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:23.719560 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.719499 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bb59438-f2b5-41f4-b963-03afaac575c7-config-file\") pod \"limitador-limitador-64c94fd64f-6kgrv\" (UID: \"4bb59438-f2b5-41f4-b963-03afaac575c7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:23.726630 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.726607 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7mqk\" (UniqueName: \"kubernetes.io/projected/4bb59438-f2b5-41f4-b963-03afaac575c7-kube-api-access-w7mqk\") pod \"limitador-limitador-64c94fd64f-6kgrv\" (UID: \"4bb59438-f2b5-41f4-b963-03afaac575c7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:23.805636 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.805585 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:23.933742 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:23.933719 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6kgrv"] Jul 12 18:02:23.935601 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:02:23.935575 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bb59438_f2b5_41f4_b963_03afaac575c7.slice/crio-d3dd8a050d02ab3083074bb6d22fe60b658b28fec3d711aae10061fce0046d5c WatchSource:0}: Error finding container d3dd8a050d02ab3083074bb6d22fe60b658b28fec3d711aae10061fce0046d5c: Status 404 returned error can't find the container with id d3dd8a050d02ab3083074bb6d22fe60b658b28fec3d711aae10061fce0046d5c Jul 12 18:02:24.178198 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:24.178168 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" event={"ID":"4bb59438-f2b5-41f4-b963-03afaac575c7","Type":"ContainerStarted","Data":"d3dd8a050d02ab3083074bb6d22fe60b658b28fec3d711aae10061fce0046d5c"} Jul 12 18:02:24.325892 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:24.325858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:24.427302 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:24.427274 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:27.199472 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:27.199431 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" event={"ID":"4bb59438-f2b5-41f4-b963-03afaac575c7","Type":"ContainerStarted","Data":"cb892e169b29f56019245604e9353b016bf715fe922f478e86c5bc68a918dfa1"} Jul 12 18:02:27.199978 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:27.199661 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:27.215728 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:27.215689 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" podStartSLOduration=1.60086007 podStartE2EDuration="4.215675534s" totalImagesPullingTime="2.614815464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:02:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:02:27.214858475 +0000 UTC m=+540.858701198" watchObservedRunningTime="2026-07-12 18:02:27.215675534 +0000 UTC m=+540.859518256" Jul 12 18:02:38.205668 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:38.205640 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:38.671945 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:38.671914 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6kgrv"] Jul 12 18:02:38.672107 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:38.672085 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" podUID="4bb59438-f2b5-41f4-b963-03afaac575c7" containerName="limitador" containerID="cri-o://cb892e169b29f56019245604e9353b016bf715fe922f478e86c5bc68a918dfa1" gracePeriod=30 Jul 12 18:02:39.255610 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.255527 2568 generic.go:356] "Generic (PLEG): container finished" podID="4bb59438-f2b5-41f4-b963-03afaac575c7" containerID="cb892e169b29f56019245604e9353b016bf715fe922f478e86c5bc68a918dfa1" exitCode=0 Jul 12 18:02:39.255610 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.255578 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" event={"ID":"4bb59438-f2b5-41f4-b963-03afaac575c7","Type":"ContainerDied","Data":"cb892e169b29f56019245604e9353b016bf715fe922f478e86c5bc68a918dfa1"} Jul 12 18:02:39.604448 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.604426 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:39.757469 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.757442 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7mqk\" (UniqueName: \"kubernetes.io/projected/4bb59438-f2b5-41f4-b963-03afaac575c7-kube-api-access-w7mqk\") pod \"4bb59438-f2b5-41f4-b963-03afaac575c7\" (UID: \"4bb59438-f2b5-41f4-b963-03afaac575c7\") " Jul 12 18:02:39.757667 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.757497 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bb59438-f2b5-41f4-b963-03afaac575c7-config-file\") pod \"4bb59438-f2b5-41f4-b963-03afaac575c7\" (UID: \"4bb59438-f2b5-41f4-b963-03afaac575c7\") " Jul 12 18:02:39.757889 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.757867 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bb59438-f2b5-41f4-b963-03afaac575c7-config-file" (OuterVolumeSpecName: "config-file") pod "4bb59438-f2b5-41f4-b963-03afaac575c7" (UID: "4bb59438-f2b5-41f4-b963-03afaac575c7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:02:39.759411 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.759392 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bb59438-f2b5-41f4-b963-03afaac575c7-kube-api-access-w7mqk" (OuterVolumeSpecName: "kube-api-access-w7mqk") pod "4bb59438-f2b5-41f4-b963-03afaac575c7" (UID: "4bb59438-f2b5-41f4-b963-03afaac575c7"). InnerVolumeSpecName "kube-api-access-w7mqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:02:39.858346 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.858274 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7mqk\" (UniqueName: \"kubernetes.io/projected/4bb59438-f2b5-41f4-b963-03afaac575c7-kube-api-access-w7mqk\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:02:39.858346 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:39.858299 2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bb59438-f2b5-41f4-b963-03afaac575c7-config-file\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:02:40.261548 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:40.261501 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" Jul 12 18:02:40.261548 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:40.261500 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6kgrv" event={"ID":"4bb59438-f2b5-41f4-b963-03afaac575c7","Type":"ContainerDied","Data":"d3dd8a050d02ab3083074bb6d22fe60b658b28fec3d711aae10061fce0046d5c"} Jul 12 18:02:40.262046 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:40.261582 2568 scope.go:117] "RemoveContainer" containerID="cb892e169b29f56019245604e9353b016bf715fe922f478e86c5bc68a918dfa1" Jul 12 18:02:40.289797 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:40.289764 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6kgrv"] Jul 12 18:02:40.292491 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:40.292470 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6kgrv"] Jul 12 18:02:40.834400 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:40.834366 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bb59438-f2b5-41f4-b963-03afaac575c7" path="/var/lib/kubelet/pods/4bb59438-f2b5-41f4-b963-03afaac575c7/volumes" Jul 12 18:02:43.049211 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.049179 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:43.532641 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.532599 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5hv74"] Jul 12 18:02:43.532944 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.532930 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bb59438-f2b5-41f4-b963-03afaac575c7" containerName="limitador" Jul 12 18:02:43.533003 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.532947 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bb59438-f2b5-41f4-b963-03afaac575c7" containerName="limitador" Jul 12 18:02:43.533040 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.533012 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bb59438-f2b5-41f4-b963-03afaac575c7" containerName="limitador" Jul 12 18:02:43.537375 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.537356 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:43.540101 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.540078 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 18:02:43.540236 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.540128 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 12 18:02:43.541098 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.541077 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 12 18:02:43.541326 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.541310 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-glkwv\"" Jul 12 18:02:43.542695 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.542676 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5hv74"] Jul 12 18:02:43.688664 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.688632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c4p9\" (UniqueName: \"kubernetes.io/projected/b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9-kube-api-access-2c4p9\") pod \"postgres-69fb4594cf-5hv74\" (UID: \"b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:43.688824 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.688685 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9-data\") pod \"postgres-69fb4594cf-5hv74\" (UID: \"b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:43.789661 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.789596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9-data\") pod \"postgres-69fb4594cf-5hv74\" (UID: \"b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:43.789661 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.789648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c4p9\" (UniqueName: \"kubernetes.io/projected/b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9-kube-api-access-2c4p9\") pod \"postgres-69fb4594cf-5hv74\" (UID: \"b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:43.789947 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.789931 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9-data\") pod \"postgres-69fb4594cf-5hv74\" (UID: \"b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:43.797725 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.797699 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c4p9\" (UniqueName: \"kubernetes.io/projected/b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9-kube-api-access-2c4p9\") pod \"postgres-69fb4594cf-5hv74\" (UID: \"b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:43.848767 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.848745 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:43.997708 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:43.997685 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5hv74"] Jul 12 18:02:43.999563 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:02:43.999534 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c3c6e3_9f22_4e74_8d48_fe4cf84bd6b9.slice/crio-7fb9158bafa2adaa3410a86c928a99cb1c1b70608a38b92a338f8fb6b060a27b WatchSource:0}: Error finding container 7fb9158bafa2adaa3410a86c928a99cb1c1b70608a38b92a338f8fb6b060a27b: Status 404 returned error can't find the container with id 7fb9158bafa2adaa3410a86c928a99cb1c1b70608a38b92a338f8fb6b060a27b Jul 12 18:02:44.092298 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:44.092204 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:44.281135 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:44.281104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" event={"ID":"b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9","Type":"ContainerStarted","Data":"7fb9158bafa2adaa3410a86c928a99cb1c1b70608a38b92a338f8fb6b060a27b"} Jul 12 18:02:49.308768 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:49.308736 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" event={"ID":"b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9","Type":"ContainerStarted","Data":"4ba391719fcc6e767d7ff9ad7db4aff51ad38151518bb2c4d98100afad6afdf8"} Jul 12 18:02:49.309221 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:49.308798 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:49.324453 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:49.324409 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" podStartSLOduration=1.540783927 podStartE2EDuration="6.324396116s" totalImagesPullingTime="4.783612189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:02:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:02:49.324274943 +0000 UTC m=+562.968117665" watchObservedRunningTime="2026-07-12 18:02:49.324396116 +0000 UTC m=+562.968238838" Jul 12 18:02:55.344851 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:55.344821 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5hv74" Jul 12 18:02:55.650761 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:55.650730 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:55.838965 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:55.838938 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:02:56.142822 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.142747 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f9fbd77c7-mx97n"] Jul 12 18:02:56.148582 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.148560 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9fbd77c7-mx97n" Jul 12 18:02:56.151378 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.151355 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bm6zs\"" Jul 12 18:02:56.151500 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.151386 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:02:56.151500 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.151359 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:02:56.152282 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.152260 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f9fbd77c7-mx97n"] Jul 12 18:02:56.198453 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.198425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82rwb\" (UniqueName: \"kubernetes.io/projected/f1648229-5265-4458-b4e3-e7dade15b873-kube-api-access-82rwb\") pod \"authorino-6f9fbd77c7-mx97n\" (UID: \"f1648229-5265-4458-b4e3-e7dade15b873\") " pod="kuadrant-system/authorino-6f9fbd77c7-mx97n" Jul 12 18:02:56.252293 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.252263 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f9fbd77c7-mx97n"] Jul 12 18:02:56.252529 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:02:56.252496 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-82rwb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f9fbd77c7-mx97n" podUID="f1648229-5265-4458-b4e3-e7dade15b873" Jul 12 18:02:56.299704 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.299673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82rwb\" (UniqueName: \"kubernetes.io/projected/f1648229-5265-4458-b4e3-e7dade15b873-kube-api-access-82rwb\") pod \"authorino-6f9fbd77c7-mx97n\" (UID: \"f1648229-5265-4458-b4e3-e7dade15b873\") " pod="kuadrant-system/authorino-6f9fbd77c7-mx97n" Jul 12 18:02:56.307454 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.307428 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82rwb\" (UniqueName: \"kubernetes.io/projected/f1648229-5265-4458-b4e3-e7dade15b873-kube-api-access-82rwb\") pod \"authorino-6f9fbd77c7-mx97n\" (UID: \"f1648229-5265-4458-b4e3-e7dade15b873\") " pod="kuadrant-system/authorino-6f9fbd77c7-mx97n" Jul 12 18:02:56.342721 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.342699 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9fbd77c7-mx97n" Jul 12 18:02:56.349969 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.349953 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9fbd77c7-mx97n" Jul 12 18:02:56.400712 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.400687 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82rwb\" (UniqueName: \"kubernetes.io/projected/f1648229-5265-4458-b4e3-e7dade15b873-kube-api-access-82rwb\") pod \"f1648229-5265-4458-b4e3-e7dade15b873\" (UID: \"f1648229-5265-4458-b4e3-e7dade15b873\") " Jul 12 18:02:56.402690 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.402666 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1648229-5265-4458-b4e3-e7dade15b873-kube-api-access-82rwb" (OuterVolumeSpecName: "kube-api-access-82rwb") pod "f1648229-5265-4458-b4e3-e7dade15b873" (UID: "f1648229-5265-4458-b4e3-e7dade15b873"). InnerVolumeSpecName "kube-api-access-82rwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:02:56.501728 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:56.501701 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82rwb\" (UniqueName: \"kubernetes.io/projected/f1648229-5265-4458-b4e3-e7dade15b873-kube-api-access-82rwb\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:02:57.346839 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:57.346811 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f9fbd77c7-mx97n" Jul 12 18:02:57.373157 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:57.373134 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f9fbd77c7-mx97n"] Jul 12 18:02:57.379710 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:57.379685 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f9fbd77c7-mx97n"] Jul 12 18:02:58.834170 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:02:58.834142 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1648229-5265-4458-b4e3-e7dade15b873" path="/var/lib/kubelet/pods/f1648229-5265-4458-b4e3-e7dade15b873/volumes" Jul 12 18:03:04.948955 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:04.948928 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:05.041818 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.041786 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:05.135320 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.135291 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:05.138821 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.138798 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-d7r2k"] Jul 12 18:03:05.161490 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.161421 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-d7r2k"] Jul 12 18:03:05.161490 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.161489 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:05.165791 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.165745 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 12 18:03:05.165791 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.165766 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 12 18:03:05.165985 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.165806 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tj8br\"" Jul 12 18:03:05.201774 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.201699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:05.257851 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.257817 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-webhook-cert\") pod \"maas-controller-5dc7598fc6-d7r2k\" (UID: \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\") " pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:05.257984 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.257893 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db22d\" (UniqueName: \"kubernetes.io/projected/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-kube-api-access-db22d\") pod \"maas-controller-5dc7598fc6-d7r2k\" (UID: \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\") " pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:05.359280 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.359249 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-webhook-cert\") pod \"maas-controller-5dc7598fc6-d7r2k\" (UID: \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\") " pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:05.359571 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.359338 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db22d\" (UniqueName: \"kubernetes.io/projected/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-kube-api-access-db22d\") pod \"maas-controller-5dc7598fc6-d7r2k\" (UID: \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\") " pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:05.361612 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.361590 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-webhook-cert\") pod \"maas-controller-5dc7598fc6-d7r2k\" (UID: \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\") " pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:05.368014 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.367986 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db22d\" (UniqueName: \"kubernetes.io/projected/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-kube-api-access-db22d\") pod \"maas-controller-5dc7598fc6-d7r2k\" (UID: \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\") " pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:05.471655 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.471555 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:05.596542 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:05.596502 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-d7r2k"] Jul 12 18:03:05.598608 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:03:05.598580 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ded10f_f00b_4a0e_a3d1_08db7ab14aa1.slice/crio-8792225cb934294ead836ae7e1a426688a1833ae085012adf6b6e2f4dd12a6a4 WatchSource:0}: Error finding container 8792225cb934294ead836ae7e1a426688a1833ae085012adf6b6e2f4dd12a6a4: Status 404 returned error can't find the container with id 8792225cb934294ead836ae7e1a426688a1833ae085012adf6b6e2f4dd12a6a4 Jul 12 18:03:06.391773 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:06.391733 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" event={"ID":"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1","Type":"ContainerStarted","Data":"8792225cb934294ead836ae7e1a426688a1833ae085012adf6b6e2f4dd12a6a4"} Jul 12 18:03:07.399164 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:07.399132 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" event={"ID":"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1","Type":"ContainerStarted","Data":"468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987"} Jul 12 18:03:07.399617 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:07.399293 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:07.417286 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:07.417261 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:07.422393 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:07.422353 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" podStartSLOduration=0.79312787 podStartE2EDuration="2.422340099s" totalImagesPullingTime="1.629212229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:03:07.421894295 +0000 UTC m=+581.065737019" watchObservedRunningTime="2026-07-12 18:03:07.422340099 +0000 UTC m=+581.066182844" Jul 12 18:03:07.514490 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:07.514405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:18.405818 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:18.405789 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:03:39.899986 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:39.899951 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:40.002942 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.002911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:40.419527 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.419483 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:40.465971 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.465939 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8"] Jul 12 18:03:40.469073 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.469057 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.472988 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.472964 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 12 18:03:40.473269 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.473248 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jx9tg\"" Jul 12 18:03:40.482819 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.482790 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8"] Jul 12 18:03:40.524926 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.524905 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:40.531058 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.531038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46857c31-9c6e-4464-9fa5-431c969821ff-openshift-service-ca\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.531114 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.531073 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46857c31-9c6e-4464-9fa5-431c969821ff-maas-api-tls\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.531114 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.531096 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdnrm\" (UniqueName: \"kubernetes.io/projected/46857c31-9c6e-4464-9fa5-431c969821ff-kube-api-access-tdnrm\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.631831 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.631804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46857c31-9c6e-4464-9fa5-431c969821ff-openshift-service-ca\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.631991 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.631844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46857c31-9c6e-4464-9fa5-431c969821ff-maas-api-tls\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.631991 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.631879 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdnrm\" (UniqueName: \"kubernetes.io/projected/46857c31-9c6e-4464-9fa5-431c969821ff-kube-api-access-tdnrm\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.632551 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.632523 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46857c31-9c6e-4464-9fa5-431c969821ff-openshift-service-ca\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.634051 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.634035 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46857c31-9c6e-4464-9fa5-431c969821ff-maas-api-tls\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.645640 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.645618 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdnrm\" (UniqueName: \"kubernetes.io/projected/46857c31-9c6e-4464-9fa5-431c969821ff-kube-api-access-tdnrm\") pod \"maas-api-7c9bb8c854-glss8\" (UID: \"46857c31-9c6e-4464-9fa5-431c969821ff\") " pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.779341 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.779260 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:40.891933 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.891901 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:40.929217 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.929196 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8"] Jul 12 18:03:40.931144 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:03:40.931109 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46857c31_9c6e_4464_9fa5_431c969821ff.slice/crio-289d3405705c1def8ff7e4bcf925ccdc6641d887ddcc201b151ed92a8b80b733 WatchSource:0}: Error finding container 289d3405705c1def8ff7e4bcf925ccdc6641d887ddcc201b151ed92a8b80b733: Status 404 returned error can't find the container with id 289d3405705c1def8ff7e4bcf925ccdc6641d887ddcc201b151ed92a8b80b733 Jul 12 18:03:40.932608 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.932590 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:03:40.987633 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:40.987600 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:41.058081 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.058005 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5"] Jul 12 18:03:41.062388 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.062372 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:41.065085 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.065063 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hrk85\"" Jul 12 18:03:41.065085 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.065084 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 12 18:03:41.070102 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.070081 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5"] Jul 12 18:03:41.136387 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.136360 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbqxf\" (UniqueName: \"kubernetes.io/projected/41ee9192-b172-4087-8abd-78fc3880e40b-kube-api-access-zbqxf\") pod \"payload-pre-processing-74584bb9f8-6rbt5\" (UID: \"41ee9192-b172-4087-8abd-78fc3880e40b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:41.136560 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.136411 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ee9192-b172-4087-8abd-78fc3880e40b-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-6rbt5\" (UID: \"41ee9192-b172-4087-8abd-78fc3880e40b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:41.237630 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.237592 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ee9192-b172-4087-8abd-78fc3880e40b-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-6rbt5\" (UID: \"41ee9192-b172-4087-8abd-78fc3880e40b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:41.237795 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.237652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbqxf\" (UniqueName: \"kubernetes.io/projected/41ee9192-b172-4087-8abd-78fc3880e40b-kube-api-access-zbqxf\") pod \"payload-pre-processing-74584bb9f8-6rbt5\" (UID: \"41ee9192-b172-4087-8abd-78fc3880e40b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:41.238323 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.238305 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ee9192-b172-4087-8abd-78fc3880e40b-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-6rbt5\" (UID: \"41ee9192-b172-4087-8abd-78fc3880e40b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:41.247075 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.247054 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbqxf\" (UniqueName: \"kubernetes.io/projected/41ee9192-b172-4087-8abd-78fc3880e40b-kube-api-access-zbqxf\") pod \"payload-pre-processing-74584bb9f8-6rbt5\" (UID: \"41ee9192-b172-4087-8abd-78fc3880e40b\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:41.375173 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.375081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:41.505634 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.505610 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5"] Jul 12 18:03:41.507165 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:03:41.507140 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ee9192_b172_4087_8abd_78fc3880e40b.slice/crio-5831ab3c2b3c49071ee03e52214ab56f2785a572d03598ad11e64ff69d90e28c WatchSource:0}: Error finding container 5831ab3c2b3c49071ee03e52214ab56f2785a572d03598ad11e64ff69d90e28c: Status 404 returned error can't find the container with id 5831ab3c2b3c49071ee03e52214ab56f2785a572d03598ad11e64ff69d90e28c Jul 12 18:03:41.557318 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.557275 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" event={"ID":"41ee9192-b172-4087-8abd-78fc3880e40b","Type":"ContainerStarted","Data":"5831ab3c2b3c49071ee03e52214ab56f2785a572d03598ad11e64ff69d90e28c"} Jul 12 18:03:41.558531 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:41.558487 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" event={"ID":"46857c31-9c6e-4464-9fa5-431c969821ff","Type":"ContainerStarted","Data":"289d3405705c1def8ff7e4bcf925ccdc6641d887ddcc201b151ed92a8b80b733"} Jul 12 18:03:43.569736 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:43.569701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" event={"ID":"46857c31-9c6e-4464-9fa5-431c969821ff","Type":"ContainerStarted","Data":"769d0467fc51bf6224e7cd993a5d63917b647de703c16bf97751985860cbaf67"} Jul 12 18:03:43.570167 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:43.569796 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:43.571244 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:43.571225 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" event={"ID":"41ee9192-b172-4087-8abd-78fc3880e40b","Type":"ContainerStarted","Data":"e7be02b28f2eadbcfbe48fd7d150e69f3e235d92f02746d5a801c2016ab71799"} Jul 12 18:03:43.571324 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:43.571270 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:03:43.586905 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:43.586862 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" podStartSLOduration=1.54553969 podStartE2EDuration="3.586850093s" totalImagesPullingTime="2.041310403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:03:43.586179823 +0000 UTC m=+617.230022546" watchObservedRunningTime="2026-07-12 18:03:43.586850093 +0000 UTC m=+617.230692815" Jul 12 18:03:43.601745 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:43.601710 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" podStartSLOduration=1.134433375 podStartE2EDuration="2.601698148s" totalImagesPullingTime="1.467264773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:03:43.600715713 +0000 UTC m=+617.244558437" watchObservedRunningTime="2026-07-12 18:03:43.601698148 +0000 UTC m=+617.245540901" Jul 12 18:03:49.581069 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:49.581000 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7c9bb8c854-glss8" Jul 12 18:03:50.574113 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:50.574063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:50.672131 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:50.672100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:03:54.577346 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:03:54.577317 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-6rbt5" Jul 12 18:04:03.931848 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:03.931811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:04:09.995736 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:09.995703 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b"] Jul 12 18:04:09.999735 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:09.999709 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.003190 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.003167 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 18:04:10.003319 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.003167 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 18:04:10.003319 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.003167 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 12 18:04:10.003319 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.003167 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p7s5j\"" Jul 12 18:04:10.008125 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.008106 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b"] Jul 12 18:04:10.025853 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.025825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:04:10.078295 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.078262 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.078411 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.078329 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.078411 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.078362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfnss\" (UniqueName: \"kubernetes.io/projected/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-kube-api-access-kfnss\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.078411 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.078399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.078549 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.078425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.078549 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.078457 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.179602 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.179573 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.179769 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.179637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.179769 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.179663 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfnss\" (UniqueName: \"kubernetes.io/projected/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-kube-api-access-kfnss\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.179769 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.179691 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.179769 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.179711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.179769 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.179735 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.180151 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.180119 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.180151 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.180120 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.180297 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.180158 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.182013 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.181900 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.182200 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.182173 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.188036 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.188016 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfnss\" (UniqueName: \"kubernetes.io/projected/09f894d5-01ec-4f2a-a430-b4ba1268b4c8-kube-api-access-kfnss\") pod \"e2e-distinct-simulated-kserve-59975d69f9-d4x7b\" (UID: \"09f894d5-01ec-4f2a-a430-b4ba1268b4c8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.312121 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.312037 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:10.439354 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.439327 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b"] Jul 12 18:04:10.440693 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:04:10.440658 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f894d5_01ec_4f2a_a430_b4ba1268b4c8.slice/crio-d84ce4caeb9d67a3d79b153edcb8cd7006885969a8d13decef036fb63c51848c WatchSource:0}: Error finding container d84ce4caeb9d67a3d79b153edcb8cd7006885969a8d13decef036fb63c51848c: Status 404 returned error can't find the container with id d84ce4caeb9d67a3d79b153edcb8cd7006885969a8d13decef036fb63c51848c Jul 12 18:04:10.707245 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:10.707207 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" event={"ID":"09f894d5-01ec-4f2a-a430-b4ba1268b4c8","Type":"ContainerStarted","Data":"d84ce4caeb9d67a3d79b153edcb8cd7006885969a8d13decef036fb63c51848c"} Jul 12 18:04:11.713377 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:11.713340 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" event={"ID":"09f894d5-01ec-4f2a-a430-b4ba1268b4c8","Type":"ContainerStarted","Data":"89f6d9df059cea1b4a85f495144f3290968d592b7360e14e802001a50b1ab8ac"} Jul 12 18:04:11.731285 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:11.731194 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" podStartSLOduration=1.7004224730000002 podStartE2EDuration="2.731179346s" totalImagesPullingTime="1.030756873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:04:11.729827288 +0000 UTC m=+645.373670046" watchObservedRunningTime="2026-07-12 18:04:11.731179346 +0000 UTC m=+645.375022070" Jul 12 18:04:17.111911 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.111877 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76"] Jul 12 18:04:17.116372 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.116354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.118754 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.118734 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 12 18:04:17.127990 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.127969 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76"] Jul 12 18:04:17.133810 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.133789 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw8gr\" (UniqueName: \"kubernetes.io/projected/8ba97051-d449-4a1c-9f5a-59698d1f833d-kube-api-access-nw8gr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.133921 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.133824 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.133921 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.133852 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.133921 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.133871 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ba97051-d449-4a1c-9f5a-59698d1f833d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.133921 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.133893 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.134064 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.133927 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.143820 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.143790 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:04:17.234685 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.234661 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw8gr\" (UniqueName: \"kubernetes.io/projected/8ba97051-d449-4a1c-9f5a-59698d1f833d-kube-api-access-nw8gr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.234802 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.234700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.234802 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.234741 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.234802 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.234769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ba97051-d449-4a1c-9f5a-59698d1f833d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.234960 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.234793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.234960 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.234836 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.235079 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.235061 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.235154 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.235131 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.235210 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.235165 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.237054 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.237030 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ba97051-d449-4a1c-9f5a-59698d1f833d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.237443 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.237422 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ba97051-d449-4a1c-9f5a-59698d1f833d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.244879 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.244857 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw8gr\" (UniqueName: \"kubernetes.io/projected/8ba97051-d449-4a1c-9f5a-59698d1f833d-kube-api-access-nw8gr\") pod \"e2e-trlp-test-simulated-kserve-598466548d-t2w76\" (UID: \"8ba97051-d449-4a1c-9f5a-59698d1f833d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.427954 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.427907 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:17.557790 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.557767 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76"] Jul 12 18:04:17.560299 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:04:17.560274 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba97051_d449_4a1c_9f5a_59698d1f833d.slice/crio-53351bcbe18183593d3602b3c47aad8ea8fff37178482c7534ec896284114b9d WatchSource:0}: Error finding container 53351bcbe18183593d3602b3c47aad8ea8fff37178482c7534ec896284114b9d: Status 404 returned error can't find the container with id 53351bcbe18183593d3602b3c47aad8ea8fff37178482c7534ec896284114b9d Jul 12 18:04:17.742429 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:17.742408 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" event={"ID":"8ba97051-d449-4a1c-9f5a-59698d1f833d","Type":"ContainerStarted","Data":"53351bcbe18183593d3602b3c47aad8ea8fff37178482c7534ec896284114b9d"} Jul 12 18:04:18.748902 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:18.748871 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" event={"ID":"8ba97051-d449-4a1c-9f5a-59698d1f833d","Type":"ContainerStarted","Data":"6f9cbd6e68bc5c71c09135a0819459b1c797f284cba3c124f703b8989df4533b"} Jul 12 18:04:18.770526 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:18.768720 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" podStartSLOduration=1.602057282 podStartE2EDuration="1.768697026s" totalImagesPullingTime="166.639744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:04:18.76565012 +0000 UTC m=+652.409492836" watchObservedRunningTime="2026-07-12 18:04:18.768697026 +0000 UTC m=+652.412539752" Jul 12 18:04:20.312778 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:20.312740 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:20.313202 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:20.312804 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:20.325544 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:20.325500 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:20.769812 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:20.769785 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-d4x7b" Jul 12 18:04:27.428960 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:27.428923 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:27.429370 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:27.428978 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:27.441393 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:27.441367 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:27.805444 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:27.805365 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-t2w76" Jul 12 18:04:28.294425 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:28.294394 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:04:36.907578 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.907546 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz"] Jul 12 18:04:36.911329 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.911302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:36.913966 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.913943 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 12 18:04:36.921859 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.921834 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz"] Jul 12 18:04:36.942015 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.941990 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:04:36.999379 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.999357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:36.999502 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.999390 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:36.999502 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.999426 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:36.999606 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.999531 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:36.999606 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.999569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:36.999606 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:36.999594 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrz6j\" (UniqueName: \"kubernetes.io/projected/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-kube-api-access-qrz6j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.100884 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.100852 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.101016 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.100900 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.101016 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.100920 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.101016 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.100938 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrz6j\" (UniqueName: \"kubernetes.io/projected/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-kube-api-access-qrz6j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.101016 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.100977 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.101016 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.100991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.101331 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.101310 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.101406 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.101384 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.101446 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.101417 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.103092 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.103068 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.103330 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.103314 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.108876 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.108855 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrz6j\" (UniqueName: \"kubernetes.io/projected/8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf-kube-api-access-qrz6j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-24cfz\" (UID: \"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.223639 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.223563 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:37.355074 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.355051 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz"] Jul 12 18:04:37.357167 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:04:37.357139 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d5ee50f_2d9f_42cb_8465_d2ddd40a3dbf.slice/crio-ff0fb63fd33ceb87833b978b1588bfee0a68ae2820a438731dae885fad8e51c9 WatchSource:0}: Error finding container ff0fb63fd33ceb87833b978b1588bfee0a68ae2820a438731dae885fad8e51c9: Status 404 returned error can't find the container with id ff0fb63fd33ceb87833b978b1588bfee0a68ae2820a438731dae885fad8e51c9 Jul 12 18:04:37.839771 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.839676 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" event={"ID":"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf","Type":"ContainerStarted","Data":"e6c1f2471c310e00a811c6d30f4816a7df9b606854c04b66c1d6e8f515502a52"} Jul 12 18:04:37.839771 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.839753 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" event={"ID":"8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf","Type":"ContainerStarted","Data":"ff0fb63fd33ceb87833b978b1588bfee0a68ae2820a438731dae885fad8e51c9"} Jul 12 18:04:37.856709 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:37.856641 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" podStartSLOduration=1.6570464 podStartE2EDuration="1.856627056s" totalImagesPullingTime="199.580656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:04:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:04:37.856093591 +0000 UTC m=+671.499936315" watchObservedRunningTime="2026-07-12 18:04:37.856627056 +0000 UTC m=+671.500469779" Jul 12 18:04:47.223851 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:47.223816 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:47.224271 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:47.223865 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:47.236308 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:47.236286 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:47.896650 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:47.896623 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-24cfz" Jul 12 18:04:51.999857 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:04:51.999825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:05:34.708280 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:05:34.708206 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:05:34.813657 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:05:34.813625 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:05:36.523215 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:05:36.523182 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:06:15.285315 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:06:15.285271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:06:15.385720 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:06:15.385688 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:06:16.445952 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:06:16.445911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:06:16.545396 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:06:16.545360 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:06:55.311230 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:06:55.311144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:06:55.412143 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:06:55.412107 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:07:26.012008 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.011974 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:07:26.071301 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.071240 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-d7r2k"] Jul 12 18:07:26.071578 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.071548 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" podUID="d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1" containerName="manager" containerID="cri-o://468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987" gracePeriod=30 Jul 12 18:07:26.320523 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.320482 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:07:26.409937 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.409908 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db22d\" (UniqueName: \"kubernetes.io/projected/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-kube-api-access-db22d\") pod \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\" (UID: \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\") " Jul 12 18:07:26.410112 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.409946 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-webhook-cert\") pod \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\" (UID: \"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1\") " Jul 12 18:07:26.412135 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.412106 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-kube-api-access-db22d" (OuterVolumeSpecName: "kube-api-access-db22d") pod "d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1" (UID: "d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1"). InnerVolumeSpecName "kube-api-access-db22d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:07:26.412135 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.412121 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1" (UID: "d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:07:26.510698 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.510666 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db22d\" (UniqueName: \"kubernetes.io/projected/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-kube-api-access-db22d\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:07:26.510698 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.510694 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1-webhook-cert\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:07:26.640173 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.640089 2568 generic.go:356] "Generic (PLEG): container finished" podID="d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1" containerID="468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987" exitCode=0 Jul 12 18:07:26.640173 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.640136 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" Jul 12 18:07:26.640173 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.640149 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" event={"ID":"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1","Type":"ContainerDied","Data":"468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987"} Jul 12 18:07:26.640379 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.640210 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dc7598fc6-d7r2k" event={"ID":"d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1","Type":"ContainerDied","Data":"8792225cb934294ead836ae7e1a426688a1833ae085012adf6b6e2f4dd12a6a4"} Jul 12 18:07:26.640379 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.640236 2568 scope.go:117] "RemoveContainer" containerID="468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987" Jul 12 18:07:26.649985 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.649968 2568 scope.go:117] "RemoveContainer" containerID="468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987" Jul 12 18:07:26.650309 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:07:26.650287 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987\": container with ID starting with 468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987 not found: ID does not exist" containerID="468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987" Jul 12 18:07:26.650379 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.650321 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987"} err="failed to get container status \"468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987\": rpc error: code = NotFound desc = could not find container \"468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987\": container with ID starting with 468b730871e3eb591c7355244e621293ed4c767e473dd27468e24b0cb4678987 not found: ID does not exist" Jul 12 18:07:26.661685 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.661661 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-d7r2k"] Jul 12 18:07:26.666049 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.666026 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5dc7598fc6-d7r2k"] Jul 12 18:07:26.834258 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:26.834231 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1" path="/var/lib/kubelet/pods/d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1/volumes" Jul 12 18:07:27.313552 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:07:27.313494 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:14.696903 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:14.696873 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:15.927186 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:15.927154 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:17.392031 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:17.392001 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:20.533943 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:20.533869 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:20.640048 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:20.640011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:20.735917 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:20.735888 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:20.919213 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:20.919180 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:22.060207 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:22.060175 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:26.631676 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:26.631643 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:26.688900 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:26.688873 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:08:45.636491 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:08:45.636456 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:09:10.838388 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:09:10.838356 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:09:10.935116 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:09:10.935087 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:12:15.569023 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:12:15.568986 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:14:14.404638 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:14:14.404609 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:15:20.463967 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:15:20.463929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:26.789026 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:26.788993 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:35.585719 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:35.585683 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:43.855793 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:43.855762 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:48.917788 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:48.917745 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:51.810277 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:51.810239 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:51.911462 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:51.911426 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:52.222909 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:52.222879 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:57.978485 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:57.978453 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:58.042647 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:58.042611 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:58.176907 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:58.176881 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:58.237373 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:58.237255 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:59.352545 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.352481 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:59.425466 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.425429 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp"] Jul 12 18:19:59.425798 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.425778 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1" containerName="manager" Jul 12 18:19:59.425798 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.425796 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1" containerName="manager" Jul 12 18:19:59.425995 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.425890 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0ded10f-f00b-4a0e-a3d1-08db7ab14aa1" containerName="manager" Jul 12 18:19:59.428842 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.428823 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.431276 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.431258 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6a5892af-serving-cert\"" Jul 12 18:19:59.438115 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.438091 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp"] Jul 12 18:19:59.441669 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.441646 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:59.507195 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.507161 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:59.549362 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.549333 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmkgt\" (UniqueName: \"kubernetes.io/projected/9dd2f4e0-0351-44a3-af27-00f2f953a960-kube-api-access-zmkgt\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.549567 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.549376 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd2f4e0-0351-44a3-af27-00f2f953a960-maas-api-tls\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.549567 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.549462 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd2f4e0-0351-44a3-af27-00f2f953a960-openshift-service-ca\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.615065 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.614944 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:19:59.650350 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.650312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd2f4e0-0351-44a3-af27-00f2f953a960-maas-api-tls\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.650555 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.650370 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd2f4e0-0351-44a3-af27-00f2f953a960-openshift-service-ca\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.650555 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.650424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmkgt\" (UniqueName: \"kubernetes.io/projected/9dd2f4e0-0351-44a3-af27-00f2f953a960-kube-api-access-zmkgt\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.651151 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.651131 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd2f4e0-0351-44a3-af27-00f2f953a960-openshift-service-ca\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.652799 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.652782 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd2f4e0-0351-44a3-af27-00f2f953a960-maas-api-tls\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.658608 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.658585 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmkgt\" (UniqueName: \"kubernetes.io/projected/9dd2f4e0-0351-44a3-af27-00f2f953a960-kube-api-access-zmkgt\") pod \"maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.740975 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.740942 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:19:59.879374 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.879344 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp"] Jul 12 18:19:59.881078 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:19:59.881053 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dd2f4e0_0351_44a3_af27_00f2f953a960.slice/crio-ab14efc7537d4e1e7392179dd8d2097f5a899b8236c35f07b27705e111e3b482 WatchSource:0}: Error finding container ab14efc7537d4e1e7392179dd8d2097f5a899b8236c35f07b27705e111e3b482: Status 404 returned error can't find the container with id ab14efc7537d4e1e7392179dd8d2097f5a899b8236c35f07b27705e111e3b482 Jul 12 18:19:59.882945 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.882926 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:19:59.903839 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:19:59.903811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:20:00.003715 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:20:00.003688 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:20:00.273948 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:20:00.273914 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" event={"ID":"9dd2f4e0-0351-44a3-af27-00f2f953a960","Type":"ContainerStarted","Data":"ab14efc7537d4e1e7392179dd8d2097f5a899b8236c35f07b27705e111e3b482"} Jul 12 18:20:01.279928 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:20:01.279890 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" event={"ID":"9dd2f4e0-0351-44a3-af27-00f2f953a960","Type":"ContainerStarted","Data":"531e978afd5d44398257f30030a4c5bb3a8c5a7ede2c2941d570e659b1a5a4aa"} Jul 12 18:20:01.280322 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:20:01.279982 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:20:01.299643 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:20:01.299590 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" podStartSLOduration=1.894335024 podStartE2EDuration="2.299573648s" totalImagesPullingTime="405.238624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:20:01.298398721 +0000 UTC m=+1594.942241445" watchObservedRunningTime="2026-07-12 18:20:01.299573648 +0000 UTC m=+1594.943416399" Jul 12 18:20:02.020730 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:20:02.020697 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:20:07.290741 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:20:07.290704 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:21:47.552236 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:21:47.552191 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:21:47.608185 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:21:47.608152 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:22:37.660974 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:22:37.660943 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:22:51.109423 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:22:51.109390 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:22:51.210871 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:22:51.210834 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:04.464602 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:04.464568 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:04.581333 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:04.581286 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:04.962552 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:04.962498 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:05.062144 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:05.062114 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:10.004326 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.004236 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp"] Jul 12 18:23:10.004841 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.004539 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" podUID="9dd2f4e0-0351-44a3-af27-00f2f953a960" containerName="maas-api" containerID="cri-o://531e978afd5d44398257f30030a4c5bb3a8c5a7ede2c2941d570e659b1a5a4aa" gracePeriod=30 Jul 12 18:23:10.036232 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.036201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:10.199797 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.199760 2568 generic.go:356] "Generic (PLEG): container finished" podID="9dd2f4e0-0351-44a3-af27-00f2f953a960" containerID="531e978afd5d44398257f30030a4c5bb3a8c5a7ede2c2941d570e659b1a5a4aa" exitCode=0 Jul 12 18:23:10.199963 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.199831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" event={"ID":"9dd2f4e0-0351-44a3-af27-00f2f953a960","Type":"ContainerDied","Data":"531e978afd5d44398257f30030a4c5bb3a8c5a7ede2c2941d570e659b1a5a4aa"} Jul 12 18:23:10.252236 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.252211 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:23:10.276687 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.276626 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd2f4e0-0351-44a3-af27-00f2f953a960-openshift-service-ca\") pod \"9dd2f4e0-0351-44a3-af27-00f2f953a960\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " Jul 12 18:23:10.276836 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.276688 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd2f4e0-0351-44a3-af27-00f2f953a960-maas-api-tls\") pod \"9dd2f4e0-0351-44a3-af27-00f2f953a960\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " Jul 12 18:23:10.276836 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.276763 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmkgt\" (UniqueName: \"kubernetes.io/projected/9dd2f4e0-0351-44a3-af27-00f2f953a960-kube-api-access-zmkgt\") pod \"9dd2f4e0-0351-44a3-af27-00f2f953a960\" (UID: \"9dd2f4e0-0351-44a3-af27-00f2f953a960\") " Jul 12 18:23:10.277029 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.277006 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dd2f4e0-0351-44a3-af27-00f2f953a960-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9dd2f4e0-0351-44a3-af27-00f2f953a960" (UID: "9dd2f4e0-0351-44a3-af27-00f2f953a960"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:23:10.278977 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.278954 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9dd2f4e0-0351-44a3-af27-00f2f953a960-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9dd2f4e0-0351-44a3-af27-00f2f953a960" (UID: "9dd2f4e0-0351-44a3-af27-00f2f953a960"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:23:10.279074 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.278950 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dd2f4e0-0351-44a3-af27-00f2f953a960-kube-api-access-zmkgt" (OuterVolumeSpecName: "kube-api-access-zmkgt") pod "9dd2f4e0-0351-44a3-af27-00f2f953a960" (UID: "9dd2f4e0-0351-44a3-af27-00f2f953a960"). InnerVolumeSpecName "kube-api-access-zmkgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:23:10.378226 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.378188 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9dd2f4e0-0351-44a3-af27-00f2f953a960-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:23:10.378226 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.378221 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmkgt\" (UniqueName: \"kubernetes.io/projected/9dd2f4e0-0351-44a3-af27-00f2f953a960-kube-api-access-zmkgt\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:23:10.378226 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.378230 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dd2f4e0-0351-44a3-af27-00f2f953a960-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:23:10.630005 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:10.629931 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:11.205417 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:11.205385 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" event={"ID":"9dd2f4e0-0351-44a3-af27-00f2f953a960","Type":"ContainerDied","Data":"ab14efc7537d4e1e7392179dd8d2097f5a899b8236c35f07b27705e111e3b482"} Jul 12 18:23:11.205870 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:11.205439 2568 scope.go:117] "RemoveContainer" containerID="531e978afd5d44398257f30030a4c5bb3a8c5a7ede2c2941d570e659b1a5a4aa" Jul 12 18:23:11.205870 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:11.205390 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp" Jul 12 18:23:11.226213 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:11.226185 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp"] Jul 12 18:23:11.230188 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:11.230163 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6a5892af-69fdc5cf9c-g6zlp"] Jul 12 18:23:12.834530 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:12.834482 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dd2f4e0-0351-44a3-af27-00f2f953a960" path="/var/lib/kubelet/pods/9dd2f4e0-0351-44a3-af27-00f2f953a960/volumes" Jul 12 18:23:16.154693 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:16.154656 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:16.228310 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:16.228255 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:16.441679 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:16.441575 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:16.541938 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:16.541905 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:16.632057 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:16.632016 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:17.862391 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.862353 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:17.932116 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.932084 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6"] Jul 12 18:23:17.932416 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.932404 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dd2f4e0-0351-44a3-af27-00f2f953a960" containerName="maas-api" Jul 12 18:23:17.932416 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.932417 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dd2f4e0-0351-44a3-af27-00f2f953a960" containerName="maas-api" Jul 12 18:23:17.932543 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.932470 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dd2f4e0-0351-44a3-af27-00f2f953a960" containerName="maas-api" Jul 12 18:23:17.936904 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.936881 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:17.939175 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.939158 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-ad0567c9-serving-cert\"" Jul 12 18:23:17.944416 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.944391 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6"] Jul 12 18:23:17.961062 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:17.961022 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:18.044233 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.044158 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.044233 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.044195 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.044233 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.044222 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28fz9\" (UniqueName: \"kubernetes.io/projected/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-kube-api-access-28fz9\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.145166 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.145082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.145166 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.145120 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.145347 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:23:18.145228 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-serving-cert: secret "maas-api-e2e-migrate-ad0567c9-serving-cert" not found Jul 12 18:23:18.145347 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.145241 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28fz9\" (UniqueName: \"kubernetes.io/projected/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-kube-api-access-28fz9\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.145347 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:23:18.145305 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls podName:12fa1c5f-ea4c-49f5-9331-b0106c4a1904 nodeName:}" failed. No retries permitted until 2026-07-12 18:23:18.64528455 +0000 UTC m=+1792.289127262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls") pod "maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" (UID: "12fa1c5f-ea4c-49f5-9331-b0106c4a1904") : secret "maas-api-e2e-migrate-ad0567c9-serving-cert" not found Jul 12 18:23:18.145901 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.145884 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-openshift-service-ca\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.155418 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.155390 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28fz9\" (UniqueName: \"kubernetes.io/projected/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-kube-api-access-28fz9\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.519465 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.519409 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:18.618751 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.618707 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:18.649234 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.649196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.652524 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.652488 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls\") pod \"maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.849641 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.849555 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:18.982114 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:18.982089 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6"] Jul 12 18:23:18.983855 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:23:18.983834 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fa1c5f_ea4c_49f5_9331_b0106c4a1904.slice/crio-ea018ff1926adb11f37103a6ef0d48c085830a2de4f8b8e59c3ce8502cee4b87 WatchSource:0}: Error finding container ea018ff1926adb11f37103a6ef0d48c085830a2de4f8b8e59c3ce8502cee4b87: Status 404 returned error can't find the container with id ea018ff1926adb11f37103a6ef0d48c085830a2de4f8b8e59c3ce8502cee4b87 Jul 12 18:23:19.246049 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:19.246011 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" event={"ID":"12fa1c5f-ea4c-49f5-9331-b0106c4a1904","Type":"ContainerStarted","Data":"ea018ff1926adb11f37103a6ef0d48c085830a2de4f8b8e59c3ce8502cee4b87"} Jul 12 18:23:20.252331 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:20.252296 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" event={"ID":"12fa1c5f-ea4c-49f5-9331-b0106c4a1904","Type":"ContainerStarted","Data":"d7e4327974de8925f7a40705aa3293c23a7fa23b2bf50d66ec1e42af740de656"} Jul 12 18:23:20.252777 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:20.252423 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:20.274957 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:20.274880 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" podStartSLOduration=2.720722158 podStartE2EDuration="3.274861248s" totalImagesPullingTime="554.13909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:23:20.26985406 +0000 UTC m=+1793.913696782" watchObservedRunningTime="2026-07-12 18:23:20.274861248 +0000 UTC m=+1793.918703973" Jul 12 18:23:25.802298 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:25.802270 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:25.856358 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:25.856329 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:26.263054 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:26.263019 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:23:28.896824 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:28.896794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:40.154545 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:40.154489 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:23:40.209146 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:23:40.209112 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:22.506201 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:22.506163 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:22.604619 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:22.604586 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:28.403023 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.402985 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6"] Jul 12 18:24:28.403590 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.403252 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" podUID="12fa1c5f-ea4c-49f5-9331-b0106c4a1904" containerName="maas-api" containerID="cri-o://d7e4327974de8925f7a40705aa3293c23a7fa23b2bf50d66ec1e42af740de656" gracePeriod=30 Jul 12 18:24:28.429747 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.429721 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:28.585841 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.585815 2568 generic.go:356] "Generic (PLEG): container finished" podID="12fa1c5f-ea4c-49f5-9331-b0106c4a1904" containerID="d7e4327974de8925f7a40705aa3293c23a7fa23b2bf50d66ec1e42af740de656" exitCode=0 Jul 12 18:24:28.585987 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.585890 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" event={"ID":"12fa1c5f-ea4c-49f5-9331-b0106c4a1904","Type":"ContainerDied","Data":"d7e4327974de8925f7a40705aa3293c23a7fa23b2bf50d66ec1e42af740de656"} Jul 12 18:24:28.648123 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.648098 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:24:28.746296 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.746207 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28fz9\" (UniqueName: \"kubernetes.io/projected/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-kube-api-access-28fz9\") pod \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " Jul 12 18:24:28.746296 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.746264 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-openshift-service-ca\") pod \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " Jul 12 18:24:28.746296 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.746291 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls\") pod \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\" (UID: \"12fa1c5f-ea4c-49f5-9331-b0106c4a1904\") " Jul 12 18:24:28.746664 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.746636 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "12fa1c5f-ea4c-49f5-9331-b0106c4a1904" (UID: "12fa1c5f-ea4c-49f5-9331-b0106c4a1904"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:24:28.748384 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.748358 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "12fa1c5f-ea4c-49f5-9331-b0106c4a1904" (UID: "12fa1c5f-ea4c-49f5-9331-b0106c4a1904"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:24:28.748475 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.748406 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-kube-api-access-28fz9" (OuterVolumeSpecName: "kube-api-access-28fz9") pod "12fa1c5f-ea4c-49f5-9331-b0106c4a1904" (UID: "12fa1c5f-ea4c-49f5-9331-b0106c4a1904"). InnerVolumeSpecName "kube-api-access-28fz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:24:28.847790 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.847765 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:24:28.847790 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.847790 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:24:28.847957 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:28.847803 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28fz9\" (UniqueName: \"kubernetes.io/projected/12fa1c5f-ea4c-49f5-9331-b0106c4a1904-kube-api-access-28fz9\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:24:29.445913 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:29.445871 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:29.592055 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:29.592023 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" event={"ID":"12fa1c5f-ea4c-49f5-9331-b0106c4a1904","Type":"ContainerDied","Data":"ea018ff1926adb11f37103a6ef0d48c085830a2de4f8b8e59c3ce8502cee4b87"} Jul 12 18:24:29.592216 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:29.592103 2568 scope.go:117] "RemoveContainer" containerID="d7e4327974de8925f7a40705aa3293c23a7fa23b2bf50d66ec1e42af740de656" Jul 12 18:24:29.592216 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:29.592030 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6" Jul 12 18:24:29.612845 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:29.612818 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6"] Jul 12 18:24:29.616995 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:29.616972 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ad0567c9-684574d685-qpnm6"] Jul 12 18:24:30.835029 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:30.834994 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12fa1c5f-ea4c-49f5-9331-b0106c4a1904" path="/var/lib/kubelet/pods/12fa1c5f-ea4c-49f5-9331-b0106c4a1904/volumes" Jul 12 18:24:34.296737 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:34.296680 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:34.398180 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:34.398135 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:34.501660 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:34.501627 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:34.602007 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:34.601913 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:35.608746 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.608712 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:35.672250 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.672218 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c"] Jul 12 18:24:35.672673 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.672656 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12fa1c5f-ea4c-49f5-9331-b0106c4a1904" containerName="maas-api" Jul 12 18:24:35.672673 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.672674 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="12fa1c5f-ea4c-49f5-9331-b0106c4a1904" containerName="maas-api" Jul 12 18:24:35.672828 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.672776 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="12fa1c5f-ea4c-49f5-9331-b0106c4a1904" containerName="maas-api" Jul 12 18:24:35.677085 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.677065 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.679426 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.679407 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-67bdeceb-serving-cert\"" Jul 12 18:24:35.687436 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.687415 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c"] Jul 12 18:24:35.691479 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.691444 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:35.764846 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.764811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:35.806787 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.806754 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e707d57d-881d-4549-bc29-9e2228c7c298-maas-api-tls\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.806787 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.806792 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8kb2\" (UniqueName: \"kubernetes.io/projected/e707d57d-881d-4549-bc29-9e2228c7c298-kube-api-access-p8kb2\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.807041 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.806811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e707d57d-881d-4549-bc29-9e2228c7c298-openshift-service-ca\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.907723 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.907688 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e707d57d-881d-4549-bc29-9e2228c7c298-maas-api-tls\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.907929 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.907744 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8kb2\" (UniqueName: \"kubernetes.io/projected/e707d57d-881d-4549-bc29-9e2228c7c298-kube-api-access-p8kb2\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.907929 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.907776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e707d57d-881d-4549-bc29-9e2228c7c298-openshift-service-ca\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.908617 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.908592 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e707d57d-881d-4549-bc29-9e2228c7c298-openshift-service-ca\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.910235 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.910208 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e707d57d-881d-4549-bc29-9e2228c7c298-maas-api-tls\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.919983 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.919963 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8kb2\" (UniqueName: \"kubernetes.io/projected/e707d57d-881d-4549-bc29-9e2228c7c298-kube-api-access-p8kb2\") pod \"maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:35.953543 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.953493 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:35.988488 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:35.988450 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:36.140009 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:36.139917 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:36.141192 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:36.141169 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c"] Jul 12 18:24:36.143643 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:24:36.143619 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode707d57d_881d_4549_bc29_9e2228c7c298.slice/crio-f2728b8946aeed00393448ede61c40eaec68020e6a40fb189bf90bc41fc65fb6 WatchSource:0}: Error finding container f2728b8946aeed00393448ede61c40eaec68020e6a40fb189bf90bc41fc65fb6: Status 404 returned error can't find the container with id f2728b8946aeed00393448ede61c40eaec68020e6a40fb189bf90bc41fc65fb6 Jul 12 18:24:36.250140 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:36.250050 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:36.626813 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:36.626788 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" event={"ID":"e707d57d-881d-4549-bc29-9e2228c7c298","Type":"ContainerStarted","Data":"f2728b8946aeed00393448ede61c40eaec68020e6a40fb189bf90bc41fc65fb6"} Jul 12 18:24:37.633446 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:37.633411 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" event={"ID":"e707d57d-881d-4549-bc29-9e2228c7c298","Type":"ContainerStarted","Data":"ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df"} Jul 12 18:24:37.633921 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:37.633555 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:24:37.651105 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:37.651033 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" podStartSLOduration=2.191006004 podStartE2EDuration="2.6510155s" totalImagesPullingTime="460.009496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:24:37.65065036 +0000 UTC m=+1871.294493112" watchObservedRunningTime="2026-07-12 18:24:37.6510155 +0000 UTC m=+1871.294858226" Jul 12 18:24:37.832892 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:37.832859 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:24:43.644425 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:24:43.644398 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:27:51.861053 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:27:51.860968 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:27:51.950047 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:27:51.950006 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:28:11.360854 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:28:11.360821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:28:11.462346 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:28:11.462310 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:29:38.843736 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:29:38.843701 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:29:39.944240 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:29:39.944206 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:40.973725 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:40.973688 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:41.073439 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:41.073405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:46.133180 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.133142 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c"] Jul 12 18:30:46.133669 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.133405 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" podUID="e707d57d-881d-4549-bc29-9e2228c7c298" containerName="maas-api" containerID="cri-o://ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df" gracePeriod=30 Jul 12 18:30:46.160169 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.160142 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:46.384793 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.384730 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:30:46.396394 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.396369 2568 generic.go:356] "Generic (PLEG): container finished" podID="e707d57d-881d-4549-bc29-9e2228c7c298" containerID="ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df" exitCode=0 Jul 12 18:30:46.396561 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.396408 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" Jul 12 18:30:46.396561 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.396452 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" event={"ID":"e707d57d-881d-4549-bc29-9e2228c7c298","Type":"ContainerDied","Data":"ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df"} Jul 12 18:30:46.396561 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.396502 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c" event={"ID":"e707d57d-881d-4549-bc29-9e2228c7c298","Type":"ContainerDied","Data":"f2728b8946aeed00393448ede61c40eaec68020e6a40fb189bf90bc41fc65fb6"} Jul 12 18:30:46.396561 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.396529 2568 scope.go:117] "RemoveContainer" containerID="ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df" Jul 12 18:30:46.407130 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.407104 2568 scope.go:117] "RemoveContainer" containerID="ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df" Jul 12 18:30:46.407415 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:30:46.407399 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df\": container with ID starting with ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df not found: ID does not exist" containerID="ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df" Jul 12 18:30:46.407488 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.407426 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df"} err="failed to get container status \"ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df\": rpc error: code = NotFound desc = could not find container \"ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df\": container with ID starting with ff7b6dcfa96216330b4a36bed450a4d99b9c438dd8e87c3fd2f90bc5ae1a30df not found: ID does not exist" Jul 12 18:30:46.496305 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.496264 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8kb2\" (UniqueName: \"kubernetes.io/projected/e707d57d-881d-4549-bc29-9e2228c7c298-kube-api-access-p8kb2\") pod \"e707d57d-881d-4549-bc29-9e2228c7c298\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " Jul 12 18:30:46.496484 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.496318 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e707d57d-881d-4549-bc29-9e2228c7c298-openshift-service-ca\") pod \"e707d57d-881d-4549-bc29-9e2228c7c298\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " Jul 12 18:30:46.496484 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.496342 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e707d57d-881d-4549-bc29-9e2228c7c298-maas-api-tls\") pod \"e707d57d-881d-4549-bc29-9e2228c7c298\" (UID: \"e707d57d-881d-4549-bc29-9e2228c7c298\") " Jul 12 18:30:46.496738 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.496712 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e707d57d-881d-4549-bc29-9e2228c7c298-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e707d57d-881d-4549-bc29-9e2228c7c298" (UID: "e707d57d-881d-4549-bc29-9e2228c7c298"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:30:46.498474 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.498447 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e707d57d-881d-4549-bc29-9e2228c7c298-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e707d57d-881d-4549-bc29-9e2228c7c298" (UID: "e707d57d-881d-4549-bc29-9e2228c7c298"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:30:46.498474 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.498458 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e707d57d-881d-4549-bc29-9e2228c7c298-kube-api-access-p8kb2" (OuterVolumeSpecName: "kube-api-access-p8kb2") pod "e707d57d-881d-4549-bc29-9e2228c7c298" (UID: "e707d57d-881d-4549-bc29-9e2228c7c298"). InnerVolumeSpecName "kube-api-access-p8kb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:30:46.597748 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.597711 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e707d57d-881d-4549-bc29-9e2228c7c298-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.597748 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.597740 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e707d57d-881d-4549-bc29-9e2228c7c298-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.597748 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.597749 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8kb2\" (UniqueName: \"kubernetes.io/projected/e707d57d-881d-4549-bc29-9e2228c7c298-kube-api-access-p8kb2\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:30:46.717032 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.716999 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c"] Jul 12 18:30:46.722980 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.722951 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67bdeceb-97b6498ff-pd54c"] Jul 12 18:30:46.835025 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.834993 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e707d57d-881d-4549-bc29-9e2228c7c298" path="/var/lib/kubelet/pods/e707d57d-881d-4549-bc29-9e2228c7c298/volumes" Jul 12 18:30:46.984939 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:46.984848 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:52.415773 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:52.415686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:52.517033 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:52.516992 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:52.641344 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:52.641310 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:53.766101 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.766066 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:53.838028 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.837992 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8"] Jul 12 18:30:53.838959 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.838932 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e707d57d-881d-4549-bc29-9e2228c7c298" containerName="maas-api" Jul 12 18:30:53.839119 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.839106 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e707d57d-881d-4549-bc29-9e2228c7c298" containerName="maas-api" Jul 12 18:30:53.839352 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.839340 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e707d57d-881d-4549-bc29-9e2228c7c298" containerName="maas-api" Jul 12 18:30:53.845541 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.845491 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:53.848530 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.848460 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d19d5af8-serving-cert\"" Jul 12 18:30:53.849455 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.849426 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8"] Jul 12 18:30:53.861255 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.861227 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:53.927419 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.927378 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:53.960775 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.960738 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz2xw\" (UniqueName: \"kubernetes.io/projected/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-kube-api-access-hz2xw\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:53.960950 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.960781 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-maas-api-tls\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:53.960950 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:53.960850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-openshift-service-ca\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:54.028077 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.027973 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:54.061834 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.061800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-maas-api-tls\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:54.061990 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.061861 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-openshift-service-ca\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:54.061990 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.061913 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz2xw\" (UniqueName: \"kubernetes.io/projected/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-kube-api-access-hz2xw\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:54.062649 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.062631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-openshift-service-ca\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:54.064248 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.064231 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-maas-api-tls\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:54.070258 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.070232 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz2xw\" (UniqueName: \"kubernetes.io/projected/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-kube-api-access-hz2xw\") pod \"maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:54.159587 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.159552 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:30:54.289755 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.289680 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8"] Jul 12 18:30:54.292470 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:30:54.292443 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cc1ca79_4c96_467b_aa6f_b2f9ea1788f0.slice/crio-652c0fa134b52e89873745120ae3824aa2f37cb2ac3b9d218ca7eab3e623292c WatchSource:0}: Error finding container 652c0fa134b52e89873745120ae3824aa2f37cb2ac3b9d218ca7eab3e623292c: Status 404 returned error can't find the container with id 652c0fa134b52e89873745120ae3824aa2f37cb2ac3b9d218ca7eab3e623292c Jul 12 18:30:54.293613 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.293580 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:54.293876 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.293860 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:30:54.394817 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.394783 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:30:54.437025 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:54.436990 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" event={"ID":"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0","Type":"ContainerStarted","Data":"652c0fa134b52e89873745120ae3824aa2f37cb2ac3b9d218ca7eab3e623292c"} Jul 12 18:30:55.443647 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:55.443608 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" event={"ID":"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0","Type":"ContainerStarted","Data":"09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740"} Jul 12 18:30:55.444124 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:30:55.443714 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:31:01.454811 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:31:01.454782 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:31:01.470718 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:31:01.470662 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" podStartSLOduration=8.017649833 podStartE2EDuration="8.470640277s" totalImagesPullingTime="452.990444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:30:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:30:55.459614283 +0000 UTC m=+2249.103457007" watchObservedRunningTime="2026-07-12 18:31:01.470640277 +0000 UTC m=+2255.114483002" Jul 12 18:31:07.432079 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:31:07.432039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:31:07.494146 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:31:07.494097 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:31:10.814405 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:31:10.814375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:31:31.499467 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:31:31.499423 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:32:52.493690 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:32:52.493656 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:32:52.591916 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:32:52.591876 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:32:58.685003 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:32:58.684967 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:32:58.786050 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:32:58.786006 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:33:58.320620 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:33:58.320523 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:33:58.437106 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:33:58.437074 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:03.603095 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.603063 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8"] Jul 12 18:34:03.603523 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.603297 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" podUID="1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" containerName="maas-api" containerID="cri-o://09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740" gracePeriod=30 Jul 12 18:34:03.622525 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.622483 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:03.847635 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.847615 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:34:03.903940 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.903909 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-maas-api-tls\") pod \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " Jul 12 18:34:03.904091 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.903979 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-openshift-service-ca\") pod \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " Jul 12 18:34:03.904091 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.904057 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz2xw\" (UniqueName: \"kubernetes.io/projected/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-kube-api-access-hz2xw\") pod \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\" (UID: \"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0\") " Jul 12 18:34:03.904311 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.904270 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" (UID: "1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:34:03.906229 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.906207 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-kube-api-access-hz2xw" (OuterVolumeSpecName: "kube-api-access-hz2xw") pod "1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" (UID: "1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0"). InnerVolumeSpecName "kube-api-access-hz2xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:34:03.906323 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:03.906234 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" (UID: "1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:34:04.004675 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.004643 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz2xw\" (UniqueName: \"kubernetes.io/projected/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-kube-api-access-hz2xw\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:34:04.004675 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.004672 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:34:04.004675 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.004681 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:34:04.276811 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.276715 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:04.351534 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.351479 2568 generic.go:356] "Generic (PLEG): container finished" podID="1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" containerID="09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740" exitCode=0 Jul 12 18:34:04.351695 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.351548 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" Jul 12 18:34:04.351695 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.351547 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" event={"ID":"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0","Type":"ContainerDied","Data":"09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740"} Jul 12 18:34:04.351695 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.351603 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8" event={"ID":"1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0","Type":"ContainerDied","Data":"652c0fa134b52e89873745120ae3824aa2f37cb2ac3b9d218ca7eab3e623292c"} Jul 12 18:34:04.351695 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.351625 2568 scope.go:117] "RemoveContainer" containerID="09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740" Jul 12 18:34:04.360702 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.360672 2568 scope.go:117] "RemoveContainer" containerID="09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740" Jul 12 18:34:04.362876 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:34:04.362832 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740\": container with ID starting with 09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740 not found: ID does not exist" containerID="09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740" Jul 12 18:34:04.362993 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.362889 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740"} err="failed to get container status \"09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740\": rpc error: code = NotFound desc = could not find container \"09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740\": container with ID starting with 09219207498c5de3df711841e31523ae6d8256684181fc289610aa3c343ce740 not found: ID does not exist" Jul 12 18:34:04.376300 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.376273 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8"] Jul 12 18:34:04.380171 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.380146 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d19d5af8-5dc8966596-4lxd8"] Jul 12 18:34:04.835136 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:04.835106 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" path="/var/lib/kubelet/pods/1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0/volumes" Jul 12 18:34:09.740759 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:09.740723 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:09.842296 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:09.842261 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:10.212588 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:10.212554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:10.313751 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:10.313711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:10.412422 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:10.412376 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:11.860401 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:11.860359 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:14.594754 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:14.594727 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:17.554489 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:17.554457 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:17.624223 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:17.624190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:17.705369 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:17.705313 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:17.805825 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:17.805728 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:17.905950 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:17.905906 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:18.243633 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:18.243598 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:44.217070 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:44.217028 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:49.744466 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:49.744431 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:49.799210 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:49.799159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:49.884341 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:49.884304 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:34:49.984575 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:34:49.984544 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:05.788053 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:05.788014 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:11.708045 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:11.708010 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:11.768318 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:11.768284 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:11.840968 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:11.840937 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:11.943536 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:11.943470 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:22.428055 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:22.427950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:23.214368 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:23.214309 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:29.177709 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:29.177671 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:29.281191 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:29.281152 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:29.378385 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:29.378335 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:29.432809 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:29.432717 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:30.218563 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:30.218529 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:30.319036 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:30.318992 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:30.420232 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:30.420172 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:32.053047 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:32.053008 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:37.936997 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:37.936956 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:38.037180 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:38.037144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:38.630648 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:38.630615 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:44.282004 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:44.281962 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:44.382815 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:44.382782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:44.524995 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:44.524960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:44.833191 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:44.833161 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:45.124731 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:45.124638 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:47.699303 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.699263 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:47.763161 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.763126 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm"] Jul 12 18:35:47.763467 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.763450 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" containerName="maas-api" Jul 12 18:35:47.763467 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.763463 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" containerName="maas-api" Jul 12 18:35:47.763582 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.763561 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cc1ca79-4c96-467b-aa6f-b2f9ea1788f0" containerName="maas-api" Jul 12 18:35:47.766520 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.766492 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:47.768231 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.768162 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:47.768811 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.768791 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-70a9c475-serving-cert\"" Jul 12 18:35:47.778928 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.778906 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm"] Jul 12 18:35:47.831148 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.831098 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:47.877909 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.877865 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:47.878262 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.878243 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3249535-51ad-485e-967c-9a07411c8f87-openshift-service-ca\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:47.878795 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.878773 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrs9p\" (UniqueName: \"kubernetes.io/projected/a3249535-51ad-485e-967c-9a07411c8f87-kube-api-access-xrs9p\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:47.979579 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.979469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrs9p\" (UniqueName: \"kubernetes.io/projected/a3249535-51ad-485e-967c-9a07411c8f87-kube-api-access-xrs9p\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:47.979579 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.979565 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:47.979811 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.979619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3249535-51ad-485e-967c-9a07411c8f87-openshift-service-ca\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:47.979811 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:35:47.979704 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-serving-cert: secret "maas-api-e2e-mt-70a9c475-serving-cert" not found Jul 12 18:35:47.979811 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:35:47.979774 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls podName:a3249535-51ad-485e-967c-9a07411c8f87 nodeName:}" failed. No retries permitted until 2026-07-12 18:35:48.479756164 +0000 UTC m=+2542.123598868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls") pod "maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" (UID: "a3249535-51ad-485e-967c-9a07411c8f87") : secret "maas-api-e2e-mt-70a9c475-serving-cert" not found Jul 12 18:35:47.980222 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.980202 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3249535-51ad-485e-967c-9a07411c8f87-openshift-service-ca\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:47.989938 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:47.989915 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrs9p\" (UniqueName: \"kubernetes.io/projected/a3249535-51ad-485e-967c-9a07411c8f87-kube-api-access-xrs9p\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:48.235257 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:48.235159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:48.342795 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:48.342754 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:48.476976 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:48.476945 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:48.484040 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:48.484015 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:48.486547 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:48.486467 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls\") pod \"maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:48.678061 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:48.678030 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:48.807447 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:48.807421 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm"] Jul 12 18:35:48.809916 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:35:48.809878 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3249535_51ad_485e_967c_9a07411c8f87.slice/crio-55007aa652dd24a13eefb035549b3966c24d57e4c3ded3c295df35086def4e78 WatchSource:0}: Error finding container 55007aa652dd24a13eefb035549b3966c24d57e4c3ded3c295df35086def4e78: Status 404 returned error can't find the container with id 55007aa652dd24a13eefb035549b3966c24d57e4c3ded3c295df35086def4e78 Jul 12 18:35:48.843359 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:48.843330 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" event={"ID":"a3249535-51ad-485e-967c-9a07411c8f87","Type":"ContainerStarted","Data":"55007aa652dd24a13eefb035549b3966c24d57e4c3ded3c295df35086def4e78"} Jul 12 18:35:49.850361 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:49.850318 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" event={"ID":"a3249535-51ad-485e-967c-9a07411c8f87","Type":"ContainerStarted","Data":"b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b"} Jul 12 18:35:49.850793 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:49.850395 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:49.867232 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:49.867180 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" podStartSLOduration=2.53901169 podStartE2EDuration="2.867163968s" totalImagesPullingTime="328.152278ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:35:49.865555057 +0000 UTC m=+2543.509397781" watchObservedRunningTime="2026-07-12 18:35:49.867163968 +0000 UTC m=+2543.511006727" Jul 12 18:35:52.203674 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:52.203642 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:52.880263 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:52.880232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:52.975678 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:52.975639 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:53.312216 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:53.312186 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:55.860647 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:55.860617 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:35:59.058389 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:59.058348 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:35:59.124184 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:35:59.124142 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:36:35.467432 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:36:35.467393 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:37:44.153431 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:37:44.153387 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:37:44.209554 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:37:44.209518 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:37:53.355159 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:37:53.355122 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:37:53.453876 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:37:53.453841 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:37:59.897138 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:37:59.897103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:37:59.995301 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:37:59.995266 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:38:52.675521 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:38:52.675451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:38:59.779389 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:38:59.779357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:04.976542 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:04.976492 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm"] Jul 12 18:39:04.977023 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:04.976759 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" podUID="a3249535-51ad-485e-967c-9a07411c8f87" containerName="maas-api" containerID="cri-o://b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b" gracePeriod=30 Jul 12 18:39:05.008307 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.008281 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:05.230402 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.230342 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:39:05.317274 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.317247 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls\") pod \"a3249535-51ad-485e-967c-9a07411c8f87\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " Jul 12 18:39:05.317441 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.317298 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrs9p\" (UniqueName: \"kubernetes.io/projected/a3249535-51ad-485e-967c-9a07411c8f87-kube-api-access-xrs9p\") pod \"a3249535-51ad-485e-967c-9a07411c8f87\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " Jul 12 18:39:05.317441 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.317342 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3249535-51ad-485e-967c-9a07411c8f87-openshift-service-ca\") pod \"a3249535-51ad-485e-967c-9a07411c8f87\" (UID: \"a3249535-51ad-485e-967c-9a07411c8f87\") " Jul 12 18:39:05.317742 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.317713 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3249535-51ad-485e-967c-9a07411c8f87-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a3249535-51ad-485e-967c-9a07411c8f87" (UID: "a3249535-51ad-485e-967c-9a07411c8f87"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:39:05.319301 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.319274 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a3249535-51ad-485e-967c-9a07411c8f87" (UID: "a3249535-51ad-485e-967c-9a07411c8f87"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:39:05.319301 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.319277 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3249535-51ad-485e-967c-9a07411c8f87-kube-api-access-xrs9p" (OuterVolumeSpecName: "kube-api-access-xrs9p") pod "a3249535-51ad-485e-967c-9a07411c8f87" (UID: "a3249535-51ad-485e-967c-9a07411c8f87"). InnerVolumeSpecName "kube-api-access-xrs9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:39:05.418624 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.418589 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3249535-51ad-485e-967c-9a07411c8f87-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:39:05.418624 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.418623 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrs9p\" (UniqueName: \"kubernetes.io/projected/a3249535-51ad-485e-967c-9a07411c8f87-kube-api-access-xrs9p\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:39:05.418624 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.418635 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3249535-51ad-485e-967c-9a07411c8f87-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:39:05.566796 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.566685 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:05.778047 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.778014 2568 generic.go:356] "Generic (PLEG): container finished" podID="a3249535-51ad-485e-967c-9a07411c8f87" containerID="b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b" exitCode=0 Jul 12 18:39:05.778189 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.778056 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" Jul 12 18:39:05.778189 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.778087 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" event={"ID":"a3249535-51ad-485e-967c-9a07411c8f87","Type":"ContainerDied","Data":"b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b"} Jul 12 18:39:05.778189 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.778126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm" event={"ID":"a3249535-51ad-485e-967c-9a07411c8f87","Type":"ContainerDied","Data":"55007aa652dd24a13eefb035549b3966c24d57e4c3ded3c295df35086def4e78"} Jul 12 18:39:05.778189 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.778148 2568 scope.go:117] "RemoveContainer" containerID="b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b" Jul 12 18:39:05.787544 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.787528 2568 scope.go:117] "RemoveContainer" containerID="b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b" Jul 12 18:39:05.787779 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:39:05.787761 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b\": container with ID starting with b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b not found: ID does not exist" containerID="b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b" Jul 12 18:39:05.787827 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.787787 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b"} err="failed to get container status \"b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b\": rpc error: code = NotFound desc = could not find container \"b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b\": container with ID starting with b4ffbc0e81023f0dc95c0a8a3fb768da7803ce66874623186cf79bc355b2923b not found: ID does not exist" Jul 12 18:39:05.799486 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.799466 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm"] Jul 12 18:39:05.802411 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:05.802391 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-70a9c475-6b8cdfb5db-7czgm"] Jul 12 18:39:06.835884 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:06.835850 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3249535-51ad-485e-967c-9a07411c8f87" path="/var/lib/kubelet/pods/a3249535-51ad-485e-967c-9a07411c8f87/volumes" Jul 12 18:39:11.155768 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:11.155735 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:11.225243 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:11.225213 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:11.430221 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:11.430134 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:11.667244 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:11.667188 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:11.789053 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:11.788934 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:11.890814 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:11.890782 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:12.021518 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:12.021481 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:17.786392 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:17.786355 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:17.844348 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:17.844311 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:17.932692 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:17.932657 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:18.025966 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:18.025932 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:18.143052 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:18.142964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:23.849431 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:23.849395 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:23.906693 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:23.906650 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:30.628427 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:30.628389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:30.954305 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:30.954274 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:31.067528 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:31.067472 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:34.420619 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.420578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:34.475883 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.475851 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx"] Jul 12 18:39:34.476234 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.476220 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3249535-51ad-485e-967c-9a07411c8f87" containerName="maas-api" Jul 12 18:39:34.476278 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.476235 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3249535-51ad-485e-967c-9a07411c8f87" containerName="maas-api" Jul 12 18:39:34.476312 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.476299 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3249535-51ad-485e-967c-9a07411c8f87" containerName="maas-api" Jul 12 18:39:34.480821 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.480802 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.483210 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.483190 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-833acb39-serving-cert\"" Jul 12 18:39:34.489837 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.489812 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx"] Jul 12 18:39:34.509987 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.509958 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:34.570280 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.570242 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a52115-06c5-4b1c-ba4a-669e36ee3469-openshift-service-ca\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.570280 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.570282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.570477 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.570388 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brndt\" (UniqueName: \"kubernetes.io/projected/36a52115-06c5-4b1c-ba4a-669e36ee3469-kube-api-access-brndt\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.594963 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.594932 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:34.671640 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.671558 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brndt\" (UniqueName: \"kubernetes.io/projected/36a52115-06c5-4b1c-ba4a-669e36ee3469-kube-api-access-brndt\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.671640 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.671611 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a52115-06c5-4b1c-ba4a-669e36ee3469-openshift-service-ca\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.671640 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.671632 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.671918 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:39:34.671722 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-serving-cert: secret "maas-api-e2e-mt-833acb39-serving-cert" not found Jul 12 18:39:34.671918 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:39:34.671772 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls podName:36a52115-06c5-4b1c-ba4a-669e36ee3469 nodeName:}" failed. No retries permitted until 2026-07-12 18:39:35.171757404 +0000 UTC m=+2768.815600105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls") pod "maas-api-e2e-mt-833acb39-5b78444876-4r7tx" (UID: "36a52115-06c5-4b1c-ba4a-669e36ee3469") : secret "maas-api-e2e-mt-833acb39-serving-cert" not found Jul 12 18:39:34.672270 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.672250 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a52115-06c5-4b1c-ba4a-669e36ee3469-openshift-service-ca\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.690088 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.690058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brndt\" (UniqueName: \"kubernetes.io/projected/36a52115-06c5-4b1c-ba4a-669e36ee3469-kube-api-access-brndt\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:34.747812 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:34.747779 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:35.060095 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:35.059991 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:35.161030 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:35.161002 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:35.176257 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:35.176215 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:35.178597 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:35.178578 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls\") pod \"maas-api-e2e-mt-833acb39-5b78444876-4r7tx\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:35.393366 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:35.393271 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:35.541465 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:39:35.541364 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a52115_06c5_4b1c_ba4a_669e36ee3469.slice/crio-75a4aef9ad90b9885c36307df7a7fcc430252336303154b6f399cfa6d54af7ad WatchSource:0}: Error finding container 75a4aef9ad90b9885c36307df7a7fcc430252336303154b6f399cfa6d54af7ad: Status 404 returned error can't find the container with id 75a4aef9ad90b9885c36307df7a7fcc430252336303154b6f399cfa6d54af7ad Jul 12 18:39:35.543260 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:35.543243 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:39:35.545814 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:35.545789 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx"] Jul 12 18:39:35.921975 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:35.921942 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" event={"ID":"36a52115-06c5-4b1c-ba4a-669e36ee3469","Type":"ContainerStarted","Data":"75a4aef9ad90b9885c36307df7a7fcc430252336303154b6f399cfa6d54af7ad"} Jul 12 18:39:36.927862 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:36.927824 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" event={"ID":"36a52115-06c5-4b1c-ba4a-669e36ee3469","Type":"ContainerStarted","Data":"b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9"} Jul 12 18:39:36.928267 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:36.927992 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:36.947686 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:36.947643 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" podStartSLOduration=2.294898734 podStartE2EDuration="2.947628903s" totalImagesPullingTime="652.730169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:39:36.946369712 +0000 UTC m=+2770.590212435" watchObservedRunningTime="2026-07-12 18:39:36.947628903 +0000 UTC m=+2770.591471625" Jul 12 18:39:38.907958 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:38.907914 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:39:42.938243 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:42.938215 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:39:54.438005 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:39:54.437918 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:42:11.828105 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:42:11.828069 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:42:23.678545 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:42:23.678484 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:42:54.005428 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:42:54.005340 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:42:54.105954 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:42:54.105917 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:43:26.858080 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:43:26.858044 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:43:26.958955 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:43:26.958920 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:44:26.980959 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:44:26.980889 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:45:33.111866 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.111831 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tn4bb"] Jul 12 18:45:33.115045 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.115029 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:33.118392 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.118370 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-nkxfm\"" Jul 12 18:45:33.118485 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.118370 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:45:33.119343 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.119325 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:45:33.135718 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.135701 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tn4bb"] Jul 12 18:45:33.297384 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.297343 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lg4t\" (UniqueName: \"kubernetes.io/projected/8fa08470-824c-4d9c-9e6e-d565febbf959-kube-api-access-5lg4t\") pod \"kuadrant-operator-catalog-tn4bb\" (UID: \"8fa08470-824c-4d9c-9e6e-d565febbf959\") " pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:33.398609 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.398577 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lg4t\" (UniqueName: \"kubernetes.io/projected/8fa08470-824c-4d9c-9e6e-d565febbf959-kube-api-access-5lg4t\") pod \"kuadrant-operator-catalog-tn4bb\" (UID: \"8fa08470-824c-4d9c-9e6e-d565febbf959\") " pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:33.406820 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.406792 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lg4t\" (UniqueName: \"kubernetes.io/projected/8fa08470-824c-4d9c-9e6e-d565febbf959-kube-api-access-5lg4t\") pod \"kuadrant-operator-catalog-tn4bb\" (UID: \"8fa08470-824c-4d9c-9e6e-d565febbf959\") " pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:33.423704 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.423680 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:33.552273 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.552185 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tn4bb"] Jul 12 18:45:33.555064 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:45:33.555036 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa08470_824c_4d9c_9e6e_d565febbf959.slice/crio-6a05754e9f872e4e71733938d9351473991ed095913bddefc73d9395e9c90e47 WatchSource:0}: Error finding container 6a05754e9f872e4e71733938d9351473991ed095913bddefc73d9395e9c90e47: Status 404 returned error can't find the container with id 6a05754e9f872e4e71733938d9351473991ed095913bddefc73d9395e9c90e47 Jul 12 18:45:33.556329 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.556308 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:45:33.638048 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:33.638018 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" event={"ID":"8fa08470-824c-4d9c-9e6e-d565febbf959","Type":"ContainerStarted","Data":"6a05754e9f872e4e71733938d9351473991ed095913bddefc73d9395e9c90e47"} Jul 12 18:45:35.650241 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:35.650204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" event={"ID":"8fa08470-824c-4d9c-9e6e-d565febbf959","Type":"ContainerStarted","Data":"871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39"} Jul 12 18:45:35.666578 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:35.666529 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" podStartSLOduration=1.119148266 podStartE2EDuration="2.666498078s" totalImagesPullingTime="1.547349812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:45:35.664911581 +0000 UTC m=+3129.308754304" watchObservedRunningTime="2026-07-12 18:45:35.666498078 +0000 UTC m=+3129.310340801" Jul 12 18:45:43.423890 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:43.423861 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:43.424290 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:43.423924 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:43.446776 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:43.446751 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:43.708950 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:43.708864 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:43.742029 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:43.742000 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tn4bb"] Jul 12 18:45:45.696084 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:45.696024 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" podUID="8fa08470-824c-4d9c-9e6e-d565febbf959" containerName="registry-server" containerID="cri-o://871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39" gracePeriod=2 Jul 12 18:45:45.943126 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:45.943100 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:45.999607 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:45.999527 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lg4t\" (UniqueName: \"kubernetes.io/projected/8fa08470-824c-4d9c-9e6e-d565febbf959-kube-api-access-5lg4t\") pod \"8fa08470-824c-4d9c-9e6e-d565febbf959\" (UID: \"8fa08470-824c-4d9c-9e6e-d565febbf959\") " Jul 12 18:45:46.001494 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.001471 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa08470-824c-4d9c-9e6e-d565febbf959-kube-api-access-5lg4t" (OuterVolumeSpecName: "kube-api-access-5lg4t") pod "8fa08470-824c-4d9c-9e6e-d565febbf959" (UID: "8fa08470-824c-4d9c-9e6e-d565febbf959"). InnerVolumeSpecName "kube-api-access-5lg4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:45:46.100078 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.100048 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lg4t\" (UniqueName: \"kubernetes.io/projected/8fa08470-824c-4d9c-9e6e-d565febbf959-kube-api-access-5lg4t\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:45:46.702621 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.702587 2568 generic.go:356] "Generic (PLEG): container finished" podID="8fa08470-824c-4d9c-9e6e-d565febbf959" containerID="871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39" exitCode=0 Jul 12 18:45:46.703046 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.702630 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" Jul 12 18:45:46.703046 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.702646 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" event={"ID":"8fa08470-824c-4d9c-9e6e-d565febbf959","Type":"ContainerDied","Data":"871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39"} Jul 12 18:45:46.703046 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.702683 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tn4bb" event={"ID":"8fa08470-824c-4d9c-9e6e-d565febbf959","Type":"ContainerDied","Data":"6a05754e9f872e4e71733938d9351473991ed095913bddefc73d9395e9c90e47"} Jul 12 18:45:46.703046 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.702703 2568 scope.go:117] "RemoveContainer" containerID="871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39" Jul 12 18:45:46.711879 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.711864 2568 scope.go:117] "RemoveContainer" containerID="871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39" Jul 12 18:45:46.712119 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:45:46.712100 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39\": container with ID starting with 871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39 not found: ID does not exist" containerID="871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39" Jul 12 18:45:46.712164 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.712127 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39"} err="failed to get container status \"871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39\": rpc error: code = NotFound desc = could not find container \"871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39\": container with ID starting with 871dfb73bf2529279965a7e1c591c5992fc6d45ae3aef15044725169fcce4c39 not found: ID does not exist" Jul 12 18:45:46.724694 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.724668 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tn4bb"] Jul 12 18:45:46.726582 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.726562 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tn4bb"] Jul 12 18:45:46.834974 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:45:46.834944 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fa08470-824c-4d9c-9e6e-d565febbf959" path="/var/lib/kubelet/pods/8fa08470-824c-4d9c-9e6e-d565febbf959/volumes" Jul 12 18:46:21.837970 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:46:21.837893 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:46:26.977144 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:46:26.977109 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:46:32.063081 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:46:32.063047 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:08.293165 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:08.293133 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:08.842723 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:08.842689 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx"] Jul 12 18:47:08.842913 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:08.842892 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" podUID="36a52115-06c5-4b1c-ba4a-669e36ee3469" containerName="maas-api" containerID="cri-o://b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9" gracePeriod=30 Jul 12 18:47:08.862712 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:08.862690 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:09.026896 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.026867 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:09.083810 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.083789 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:47:09.092352 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.092327 2568 generic.go:356] "Generic (PLEG): container finished" podID="36a52115-06c5-4b1c-ba4a-669e36ee3469" containerID="b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9" exitCode=0 Jul 12 18:47:09.092463 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.092376 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" Jul 12 18:47:09.092463 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.092408 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" event={"ID":"36a52115-06c5-4b1c-ba4a-669e36ee3469","Type":"ContainerDied","Data":"b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9"} Jul 12 18:47:09.092463 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.092458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx" event={"ID":"36a52115-06c5-4b1c-ba4a-669e36ee3469","Type":"ContainerDied","Data":"75a4aef9ad90b9885c36307df7a7fcc430252336303154b6f399cfa6d54af7ad"} Jul 12 18:47:09.092589 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.092480 2568 scope.go:117] "RemoveContainer" containerID="b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9" Jul 12 18:47:09.101227 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.101191 2568 scope.go:117] "RemoveContainer" containerID="b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9" Jul 12 18:47:09.101550 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:47:09.101493 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9\": container with ID starting with b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9 not found: ID does not exist" containerID="b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9" Jul 12 18:47:09.101550 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.101542 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9"} err="failed to get container status \"b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9\": rpc error: code = NotFound desc = could not find container \"b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9\": container with ID starting with b4e1628d5a810e609bb59ca5238e268cc05b4eae91bd9e5521a9f90678b437d9 not found: ID does not exist" Jul 12 18:47:09.157828 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.157795 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls\") pod \"36a52115-06c5-4b1c-ba4a-669e36ee3469\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " Jul 12 18:47:09.157828 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.157829 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brndt\" (UniqueName: \"kubernetes.io/projected/36a52115-06c5-4b1c-ba4a-669e36ee3469-kube-api-access-brndt\") pod \"36a52115-06c5-4b1c-ba4a-669e36ee3469\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " Jul 12 18:47:09.158043 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.158024 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a52115-06c5-4b1c-ba4a-669e36ee3469-openshift-service-ca\") pod \"36a52115-06c5-4b1c-ba4a-669e36ee3469\" (UID: \"36a52115-06c5-4b1c-ba4a-669e36ee3469\") " Jul 12 18:47:09.158343 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.158320 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a52115-06c5-4b1c-ba4a-669e36ee3469-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36a52115-06c5-4b1c-ba4a-669e36ee3469" (UID: "36a52115-06c5-4b1c-ba4a-669e36ee3469"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:47:09.159923 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.159890 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36a52115-06c5-4b1c-ba4a-669e36ee3469" (UID: "36a52115-06c5-4b1c-ba4a-669e36ee3469"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:47:09.160016 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.159929 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a52115-06c5-4b1c-ba4a-669e36ee3469-kube-api-access-brndt" (OuterVolumeSpecName: "kube-api-access-brndt") pod "36a52115-06c5-4b1c-ba4a-669e36ee3469" (UID: "36a52115-06c5-4b1c-ba4a-669e36ee3469"). InnerVolumeSpecName "kube-api-access-brndt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:47:09.258634 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.258604 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a52115-06c5-4b1c-ba4a-669e36ee3469-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:47:09.258634 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.258633 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brndt\" (UniqueName: \"kubernetes.io/projected/36a52115-06c5-4b1c-ba4a-669e36ee3469-kube-api-access-brndt\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:47:09.258814 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.258642 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a52115-06c5-4b1c-ba4a-669e36ee3469-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:47:09.415042 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.415018 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx"] Jul 12 18:47:09.416863 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:09.416841 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-833acb39-5b78444876-4r7tx"] Jul 12 18:47:10.834495 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:10.834462 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36a52115-06c5-4b1c-ba4a-669e36ee3469" path="/var/lib/kubelet/pods/36a52115-06c5-4b1c-ba4a-669e36ee3469/volumes" Jul 12 18:47:14.546912 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:14.546878 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:37.823663 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:37.823623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:37.924590 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:37.924553 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:38.735974 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:38.735942 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:38.833776 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:38.833735 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:38.933488 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:38.933449 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:39.659930 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:39.659882 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:39.759477 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:39.759439 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:39.859239 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:39.859205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:41.155662 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:41.155629 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:46.913017 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:46.912986 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:47.013145 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:47.013108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:47.580577 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:47.580551 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:53.301769 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:53.301729 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:53.364704 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:53.364675 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:53.453002 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:53.452964 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:53.557177 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:53.557100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:54.676887 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:54.676845 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:47:54.775253 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:47:54.775218 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:03.839826 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:03.839790 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:03.987129 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:03.987090 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:04.326106 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:04.326073 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:19.235800 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:19.235765 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:25.262614 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:25.262577 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:30.945065 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:30.945033 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:31.045289 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:31.045257 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:31.274494 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:31.274414 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:31.583063 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:31.582982 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:31.890400 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:31.890311 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:34.108617 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.108580 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:34.208719 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.208686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:34.305797 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.305770 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5"] Jul 12 18:48:34.306080 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.306069 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a52115-06c5-4b1c-ba4a-669e36ee3469" containerName="maas-api" Jul 12 18:48:34.306125 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.306083 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a52115-06c5-4b1c-ba4a-669e36ee3469" containerName="maas-api" Jul 12 18:48:34.306125 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.306104 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa08470-824c-4d9c-9e6e-d565febbf959" containerName="registry-server" Jul 12 18:48:34.306125 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.306111 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa08470-824c-4d9c-9e6e-d565febbf959" containerName="registry-server" Jul 12 18:48:34.306218 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.306168 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a52115-06c5-4b1c-ba4a-669e36ee3469" containerName="maas-api" Jul 12 18:48:34.306218 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.306176 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fa08470-824c-4d9c-9e6e-d565febbf959" containerName="registry-server" Jul 12 18:48:34.309235 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.309216 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.312132 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.312110 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-5beeb3-serving-cert\"" Jul 12 18:48:34.317973 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.317951 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5"] Jul 12 18:48:34.345976 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.345950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:34.376589 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.376487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn4xk\" (UniqueName: \"kubernetes.io/projected/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-kube-api-access-gn4xk\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.376589 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.376546 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-maas-api-tls\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.376757 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.376607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.440756 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.440725 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:34.477992 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.477954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn4xk\" (UniqueName: \"kubernetes.io/projected/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-kube-api-access-gn4xk\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.477992 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.477992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-maas-api-tls\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.478197 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.478011 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.478607 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.478588 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-openshift-service-ca\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.480456 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.480434 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-maas-api-tls\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.485681 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.485659 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn4xk\" (UniqueName: \"kubernetes.io/projected/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-kube-api-access-gn4xk\") pod \"maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.621082 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.621050 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:34.721550 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.721494 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:34.758477 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.758453 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5"] Jul 12 18:48:34.760362 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:48:34.760338 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fdd1d70_9a78_489e_9fc1_918ed5fc3eff.slice/crio-eae0f085ee3a25fd2e3301e494934c962dbbf5491920d1fe2b7f041d39f88ca1 WatchSource:0}: Error finding container eae0f085ee3a25fd2e3301e494934c962dbbf5491920d1fe2b7f041d39f88ca1: Status 404 returned error can't find the container with id eae0f085ee3a25fd2e3301e494934c962dbbf5491920d1fe2b7f041d39f88ca1 Jul 12 18:48:34.823072 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:34.823038 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:35.502195 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:35.502155 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" event={"ID":"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff","Type":"ContainerStarted","Data":"e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364"} Jul 12 18:48:35.502605 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:35.502249 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:35.502605 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:35.502276 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" event={"ID":"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff","Type":"ContainerStarted","Data":"eae0f085ee3a25fd2e3301e494934c962dbbf5491920d1fe2b7f041d39f88ca1"} Jul 12 18:48:35.519019 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:35.518917 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" podStartSLOduration=1.016927259 podStartE2EDuration="1.518903491s" totalImagesPullingTime="501.976232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:48:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:48:35.517639202 +0000 UTC m=+3309.161481926" watchObservedRunningTime="2026-07-12 18:48:35.518903491 +0000 UTC m=+3309.162746213" Jul 12 18:48:38.502874 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:38.502840 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:38.813086 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:38.812998 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:39.099110 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:39.099034 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:39.199826 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:39.199799 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:41.513636 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:41.513612 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:48:42.049637 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.049599 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:42.139449 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.139414 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:42.211135 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.211098 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7"] Jul 12 18:48:42.215092 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.215071 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.217297 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.217277 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d2f6a5-serving-cert\"" Jul 12 18:48:42.222477 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.222454 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7"] Jul 12 18:48:42.250595 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.250568 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:42.347826 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.347742 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/975467fe-8f2b-4a77-81e1-7397522fa682-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.347980 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.347868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/975467fe-8f2b-4a77-81e1-7397522fa682-maas-api-tls\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.347980 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.347895 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj6fj\" (UniqueName: \"kubernetes.io/projected/975467fe-8f2b-4a77-81e1-7397522fa682-kube-api-access-lj6fj\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.400055 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.400025 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:42.449222 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.449184 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/975467fe-8f2b-4a77-81e1-7397522fa682-maas-api-tls\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.449222 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.449226 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj6fj\" (UniqueName: \"kubernetes.io/projected/975467fe-8f2b-4a77-81e1-7397522fa682-kube-api-access-lj6fj\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.449419 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.449261 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/975467fe-8f2b-4a77-81e1-7397522fa682-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.449995 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.449972 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/975467fe-8f2b-4a77-81e1-7397522fa682-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.451866 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.451844 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/975467fe-8f2b-4a77-81e1-7397522fa682-maas-api-tls\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.462223 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.462205 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj6fj\" (UniqueName: \"kubernetes.io/projected/975467fe-8f2b-4a77-81e1-7397522fa682-kube-api-access-lj6fj\") pod \"maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.528559 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.528524 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:42.662604 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.662577 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7"] Jul 12 18:48:42.664865 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:48:42.664837 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975467fe_8f2b_4a77_81e1_7397522fa682.slice/crio-1c6f1d8d5e341ffbe384677a450f1ce41404aac9948d7b1c169aa3576db0220f WatchSource:0}: Error finding container 1c6f1d8d5e341ffbe384677a450f1ce41404aac9948d7b1c169aa3576db0220f: Status 404 returned error can't find the container with id 1c6f1d8d5e341ffbe384677a450f1ce41404aac9948d7b1c169aa3576db0220f Jul 12 18:48:42.680735 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.680712 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:42.784409 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:42.784375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:43.542942 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:43.542910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" event={"ID":"975467fe-8f2b-4a77-81e1-7397522fa682","Type":"ContainerStarted","Data":"77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01"} Jul 12 18:48:43.543367 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:43.542962 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" event={"ID":"975467fe-8f2b-4a77-81e1-7397522fa682","Type":"ContainerStarted","Data":"1c6f1d8d5e341ffbe384677a450f1ce41404aac9948d7b1c169aa3576db0220f"} Jul 12 18:48:43.543367 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:43.542996 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:43.560681 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:43.560629 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" podStartSLOduration=1.268187856 podStartE2EDuration="1.5606139s" totalImagesPullingTime="292.426044ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:48:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:48:43.559222605 +0000 UTC m=+3317.203065324" watchObservedRunningTime="2026-07-12 18:48:43.5606139 +0000 UTC m=+3317.204456623" Jul 12 18:48:46.536663 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:46.536617 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:49.553303 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:49.553232 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:48:55.660919 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:55.660878 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:48:55.766819 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:48:55.766786 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:49:03.348837 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:49:03.348802 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:49:03.413645 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:49:03.413594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:49:06.842704 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:49:06.842676 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:49:22.032321 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:49:22.032289 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:49:44.933107 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:49:44.933077 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:49:45.824443 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:49:45.824412 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:49:46.422158 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:49:46.422128 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:49:46.523662 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:49:46.523634 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:50:45.566799 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:50:45.566768 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:50:45.624655 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:50:45.624627 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:46.329613 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:46.329578 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:51.686057 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:51.686022 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5"] Jul 12 18:52:51.686612 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:51.686497 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" podUID="7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" containerName="maas-api" containerID="cri-o://e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364" gracePeriod=30 Jul 12 18:52:51.709994 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:51.709950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:51.945144 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:51.945078 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:52:51.949977 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:51.949954 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:52.096763 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.096723 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn4xk\" (UniqueName: \"kubernetes.io/projected/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-kube-api-access-gn4xk\") pod \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " Jul 12 18:52:52.096947 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.096819 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-maas-api-tls\") pod \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " Jul 12 18:52:52.096947 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.096878 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-openshift-service-ca\") pod \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\" (UID: \"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff\") " Jul 12 18:52:52.097257 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.097234 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" (UID: "7fdd1d70-9a78-489e-9fc1-918ed5fc3eff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:52:52.098915 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.098889 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-kube-api-access-gn4xk" (OuterVolumeSpecName: "kube-api-access-gn4xk") pod "7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" (UID: "7fdd1d70-9a78-489e-9fc1-918ed5fc3eff"). InnerVolumeSpecName "kube-api-access-gn4xk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:52:52.098915 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.098901 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" (UID: "7fdd1d70-9a78-489e-9fc1-918ed5fc3eff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:52:52.198187 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.198108 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:52:52.198187 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.198134 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:52:52.198187 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.198144 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn4xk\" (UniqueName: \"kubernetes.io/projected/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff-kube-api-access-gn4xk\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:52:52.763225 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.763187 2568 generic.go:356] "Generic (PLEG): container finished" podID="7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" containerID="e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364" exitCode=0 Jul 12 18:52:52.763666 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.763243 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" Jul 12 18:52:52.763666 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.763269 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" event={"ID":"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff","Type":"ContainerDied","Data":"e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364"} Jul 12 18:52:52.763666 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.763347 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5" event={"ID":"7fdd1d70-9a78-489e-9fc1-918ed5fc3eff","Type":"ContainerDied","Data":"eae0f085ee3a25fd2e3301e494934c962dbbf5491920d1fe2b7f041d39f88ca1"} Jul 12 18:52:52.763666 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.763370 2568 scope.go:117] "RemoveContainer" containerID="e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364" Jul 12 18:52:52.772397 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.772382 2568 scope.go:117] "RemoveContainer" containerID="e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364" Jul 12 18:52:52.772663 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:52:52.772645 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364\": container with ID starting with e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364 not found: ID does not exist" containerID="e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364" Jul 12 18:52:52.772722 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.772672 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364"} err="failed to get container status \"e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364\": rpc error: code = NotFound desc = could not find container \"e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364\": container with ID starting with e6c0657a429fdd168571978b3f8b5e9ebd4e3da047926ce9b28b92c5167a5364 not found: ID does not exist" Jul 12 18:52:52.782838 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.782817 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5"] Jul 12 18:52:52.787035 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.787014 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-5beeb3-5d7cb97b6d-84dp5"] Jul 12 18:52:52.835049 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:52.835018 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" path="/var/lib/kubelet/pods/7fdd1d70-9a78-489e-9fc1-918ed5fc3eff/volumes" Jul 12 18:52:55.295691 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:55.295656 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:55.397493 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:55.397447 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:57.565446 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:57.565410 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:57.665165 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:57.665132 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:57.834647 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:57.834571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:58.071478 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:58.071441 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:58.830208 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:58.830178 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:52:58.929993 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:52:58.929954 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:55:08.950595 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:55:08.950489 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:55:58.574005 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:55:58.573962 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:04.222939 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.222896 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7"] Jul 12 18:56:04.223648 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.223093 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" podUID="975467fe-8f2b-4a77-81e1-7397522fa682" containerName="maas-api" containerID="cri-o://77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01" gracePeriod=30 Jul 12 18:56:04.251496 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.251470 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:04.468644 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.468621 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:56:04.485481 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.485401 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:04.540634 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.540598 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/975467fe-8f2b-4a77-81e1-7397522fa682-openshift-service-ca\") pod \"975467fe-8f2b-4a77-81e1-7397522fa682\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " Jul 12 18:56:04.540808 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.540654 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/975467fe-8f2b-4a77-81e1-7397522fa682-maas-api-tls\") pod \"975467fe-8f2b-4a77-81e1-7397522fa682\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " Jul 12 18:56:04.540854 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.540823 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj6fj\" (UniqueName: \"kubernetes.io/projected/975467fe-8f2b-4a77-81e1-7397522fa682-kube-api-access-lj6fj\") pod \"975467fe-8f2b-4a77-81e1-7397522fa682\" (UID: \"975467fe-8f2b-4a77-81e1-7397522fa682\") " Jul 12 18:56:04.541019 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.540993 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975467fe-8f2b-4a77-81e1-7397522fa682-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "975467fe-8f2b-4a77-81e1-7397522fa682" (UID: "975467fe-8f2b-4a77-81e1-7397522fa682"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:56:04.541117 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.541100 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/975467fe-8f2b-4a77-81e1-7397522fa682-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:56:04.542775 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.542744 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975467fe-8f2b-4a77-81e1-7397522fa682-kube-api-access-lj6fj" (OuterVolumeSpecName: "kube-api-access-lj6fj") pod "975467fe-8f2b-4a77-81e1-7397522fa682" (UID: "975467fe-8f2b-4a77-81e1-7397522fa682"). InnerVolumeSpecName "kube-api-access-lj6fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:56:04.542775 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.542756 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/975467fe-8f2b-4a77-81e1-7397522fa682-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "975467fe-8f2b-4a77-81e1-7397522fa682" (UID: "975467fe-8f2b-4a77-81e1-7397522fa682"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:56:04.641572 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.641544 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/975467fe-8f2b-4a77-81e1-7397522fa682-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:56:04.641572 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.641571 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj6fj\" (UniqueName: \"kubernetes.io/projected/975467fe-8f2b-4a77-81e1-7397522fa682-kube-api-access-lj6fj\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:56:04.693215 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.693180 2568 generic.go:356] "Generic (PLEG): container finished" podID="975467fe-8f2b-4a77-81e1-7397522fa682" containerID="77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01" exitCode=0 Jul 12 18:56:04.693390 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.693222 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" Jul 12 18:56:04.693390 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.693273 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" event={"ID":"975467fe-8f2b-4a77-81e1-7397522fa682","Type":"ContainerDied","Data":"77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01"} Jul 12 18:56:04.693390 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.693329 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7" event={"ID":"975467fe-8f2b-4a77-81e1-7397522fa682","Type":"ContainerDied","Data":"1c6f1d8d5e341ffbe384677a450f1ce41404aac9948d7b1c169aa3576db0220f"} Jul 12 18:56:04.693390 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.693359 2568 scope.go:117] "RemoveContainer" containerID="77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01" Jul 12 18:56:04.702577 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.702561 2568 scope.go:117] "RemoveContainer" containerID="77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01" Jul 12 18:56:04.702808 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:56:04.702793 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01\": container with ID starting with 77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01 not found: ID does not exist" containerID="77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01" Jul 12 18:56:04.702854 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.702818 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01"} err="failed to get container status \"77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01\": rpc error: code = NotFound desc = could not find container \"77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01\": container with ID starting with 77a18a36418fec544e962737a88627358ccfe84f0ed1b3471c30dbb592ae4d01 not found: ID does not exist" Jul 12 18:56:04.714970 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.714946 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7"] Jul 12 18:56:04.717272 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.717251 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d2f6a5-66f75fb4c-84pf7"] Jul 12 18:56:04.835134 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:04.835041 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="975467fe-8f2b-4a77-81e1-7397522fa682" path="/var/lib/kubelet/pods/975467fe-8f2b-4a77-81e1-7397522fa682/volumes" Jul 12 18:56:10.151945 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:10.151901 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:10.252771 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:10.252740 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:10.420118 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:10.420020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:10.639694 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:10.639655 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:10.999335 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:10.999300 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:11.300601 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:11.300516 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:11.400484 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:11.400416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:14.387133 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.387050 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:14.470219 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.470148 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh"] Jul 12 18:56:14.470475 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.470463 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="975467fe-8f2b-4a77-81e1-7397522fa682" containerName="maas-api" Jul 12 18:56:14.470556 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.470476 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="975467fe-8f2b-4a77-81e1-7397522fa682" containerName="maas-api" Jul 12 18:56:14.470556 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.470518 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" containerName="maas-api" Jul 12 18:56:14.470556 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.470526 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" containerName="maas-api" Jul 12 18:56:14.470734 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.470582 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fdd1d70-9a78-489e-9fc1-918ed5fc3eff" containerName="maas-api" Jul 12 18:56:14.470734 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.470590 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="975467fe-8f2b-4a77-81e1-7397522fa682" containerName="maas-api" Jul 12 18:56:14.471935 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.471904 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:14.473752 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.473728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.476089 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.476070 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-045df0-serving-cert\"" Jul 12 18:56:14.484263 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.484241 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh"] Jul 12 18:56:14.537106 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.537078 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:14.629495 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.629459 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-openshift-service-ca\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.629495 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.629497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmw24\" (UniqueName: \"kubernetes.io/projected/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-kube-api-access-hmw24\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.629742 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.629608 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-maas-api-tls\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.730798 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.730699 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-openshift-service-ca\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.730798 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.730750 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmw24\" (UniqueName: \"kubernetes.io/projected/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-kube-api-access-hmw24\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.730798 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.730790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-maas-api-tls\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.731451 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.731433 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-openshift-service-ca\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.733230 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.733207 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-maas-api-tls\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.739184 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.739159 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmw24\" (UniqueName: \"kubernetes.io/projected/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-kube-api-access-hmw24\") pod \"maas-api-e2e-api-a-045df0-78f44744d8-s8dhh\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.741901 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.741877 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:14.785564 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.785532 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:14.912207 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.912177 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:14.925981 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.925960 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh"] Jul 12 18:56:14.928094 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:56:14.928068 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39ab1bbb_c8a4_49d6_be32_ac88c77ed9bb.slice/crio-f4b0fb28eb9b75ecdd45d2bff71eaf2f84e57eb4b5a9aeaff86b63ee5a45a4fb WatchSource:0}: Error finding container f4b0fb28eb9b75ecdd45d2bff71eaf2f84e57eb4b5a9aeaff86b63ee5a45a4fb: Status 404 returned error can't find the container with id f4b0fb28eb9b75ecdd45d2bff71eaf2f84e57eb4b5a9aeaff86b63ee5a45a4fb Jul 12 18:56:14.929580 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:14.929560 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:56:15.010048 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:15.009950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:15.747222 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:15.747182 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" event={"ID":"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb","Type":"ContainerStarted","Data":"d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659"} Jul 12 18:56:15.747722 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:15.747246 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" event={"ID":"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb","Type":"ContainerStarted","Data":"f4b0fb28eb9b75ecdd45d2bff71eaf2f84e57eb4b5a9aeaff86b63ee5a45a4fb"} Jul 12 18:56:15.747722 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:15.747359 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:15.773372 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:15.773332 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" podStartSLOduration=1.398889064 podStartE2EDuration="1.77331651s" totalImagesPullingTime="374.427446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:56:15.77226029 +0000 UTC m=+3769.416103014" watchObservedRunningTime="2026-07-12 18:56:15.77331651 +0000 UTC m=+3769.417159354" Jul 12 18:56:18.806315 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:18.806226 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:19.250063 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:19.250027 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:19.557030 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:19.556956 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:20.188674 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:20.188646 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:21.757305 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:21.757275 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:56:22.286550 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.286476 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:22.398750 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.398712 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf"] Jul 12 18:56:22.400683 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.400657 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:22.402468 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.402451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.404564 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.404544 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-a249e9-serving-cert\"" Jul 12 18:56:22.411372 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.411347 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf"] Jul 12 18:56:22.493604 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.493573 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:22.500926 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.500892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-openshift-service-ca\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.501068 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.500946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsp2h\" (UniqueName: \"kubernetes.io/projected/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-kube-api-access-xsp2h\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.501068 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.501021 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-maas-api-tls\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.602138 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.602030 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-openshift-service-ca\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.602138 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.602086 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsp2h\" (UniqueName: \"kubernetes.io/projected/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-kube-api-access-xsp2h\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.602138 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.602128 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-maas-api-tls\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.602771 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.602750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-openshift-service-ca\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.604588 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.604566 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-maas-api-tls\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.610211 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.610192 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsp2h\" (UniqueName: \"kubernetes.io/projected/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-kube-api-access-xsp2h\") pod \"maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.635092 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.635067 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:22.713928 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.713896 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:22.845154 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:22.845044 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf"] Jul 12 18:56:22.847939 ip-10-0-143-136 kubenswrapper[2568]: W0712 18:56:22.847897 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2ef4d4_c870_40d3_974f_740f8ea0ce11.slice/crio-193422ce64b373d93baa47fc596f03a0e07059dd9db25ed5ece0ab7c2c9267d9 WatchSource:0}: Error finding container 193422ce64b373d93baa47fc596f03a0e07059dd9db25ed5ece0ab7c2c9267d9: Status 404 returned error can't find the container with id 193422ce64b373d93baa47fc596f03a0e07059dd9db25ed5ece0ab7c2c9267d9 Jul 12 18:56:23.008230 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:23.008202 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:23.109754 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:23.109728 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:23.793252 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:23.793217 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" event={"ID":"4b2ef4d4-c870-40d3-974f-740f8ea0ce11","Type":"ContainerStarted","Data":"798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915"} Jul 12 18:56:23.793434 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:23.793268 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" event={"ID":"4b2ef4d4-c870-40d3-974f-740f8ea0ce11","Type":"ContainerStarted","Data":"193422ce64b373d93baa47fc596f03a0e07059dd9db25ed5ece0ab7c2c9267d9"} Jul 12 18:56:23.793434 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:23.793310 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:23.809026 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:23.808957 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" podStartSLOduration=1.474776146 podStartE2EDuration="1.808940054s" totalImagesPullingTime="334.163908ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:56:23.807859454 +0000 UTC m=+3777.451702178" watchObservedRunningTime="2026-07-12 18:56:23.808940054 +0000 UTC m=+3777.452782776" Jul 12 18:56:26.778827 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:26.778793 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:29.805091 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:29.805062 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 18:56:35.129304 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:35.129269 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:35.231845 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:35.231811 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:44.243623 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:44.243587 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:56:44.341189 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:56:44.341153 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:57:05.052957 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:57:05.052929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:57:05.344379 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:57:05.344295 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:57:05.445350 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:57:05.445315 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:57:55.935411 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:57:55.935327 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:57:57.009595 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:57:57.009561 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:58:05.341944 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:58:05.341909 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:58:05.443472 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:58:05.443441 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:35.089224 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:35.089144 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:40.939728 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:40.939695 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh"] Jul 12 18:59:40.940337 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:40.939920 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" podUID="39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" containerName="maas-api" containerID="cri-o://d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659" gracePeriod=30 Jul 12 18:59:40.957242 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:40.957194 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:41.191821 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.191741 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:59:41.204801 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.204774 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:41.323176 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.323143 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-maas-api-tls\") pod \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " Jul 12 18:59:41.323347 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.323187 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-openshift-service-ca\") pod \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " Jul 12 18:59:41.323347 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.323245 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmw24\" (UniqueName: \"kubernetes.io/projected/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-kube-api-access-hmw24\") pod \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\" (UID: \"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb\") " Jul 12 18:59:41.323664 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.323635 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" (UID: "39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:59:41.325375 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.325338 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-kube-api-access-hmw24" (OuterVolumeSpecName: "kube-api-access-hmw24") pod "39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" (UID: "39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb"). InnerVolumeSpecName "kube-api-access-hmw24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:59:41.325375 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.325346 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" (UID: "39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:59:41.424489 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.424453 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:59:41.424489 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.424483 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:59:41.424489 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.424493 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmw24\" (UniqueName: \"kubernetes.io/projected/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb-kube-api-access-hmw24\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 18:59:41.759885 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.759843 2568 generic.go:356] "Generic (PLEG): container finished" podID="39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" containerID="d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659" exitCode=0 Jul 12 18:59:41.760064 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.759892 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" Jul 12 18:59:41.760064 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.759932 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" event={"ID":"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb","Type":"ContainerDied","Data":"d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659"} Jul 12 18:59:41.760064 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.759989 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh" event={"ID":"39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb","Type":"ContainerDied","Data":"f4b0fb28eb9b75ecdd45d2bff71eaf2f84e57eb4b5a9aeaff86b63ee5a45a4fb"} Jul 12 18:59:41.760064 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.760013 2568 scope.go:117] "RemoveContainer" containerID="d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659" Jul 12 18:59:41.768835 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.768816 2568 scope.go:117] "RemoveContainer" containerID="d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659" Jul 12 18:59:41.769124 ip-10-0-143-136 kubenswrapper[2568]: E0712 18:59:41.769104 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659\": container with ID starting with d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659 not found: ID does not exist" containerID="d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659" Jul 12 18:59:41.769179 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.769134 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659"} err="failed to get container status \"d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659\": rpc error: code = NotFound desc = could not find container \"d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659\": container with ID starting with d01f4e2268f32116e8e455ded00c7a935617b15e6cf21cfcec95cb5783c81659 not found: ID does not exist" Jul 12 18:59:41.779627 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.779602 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh"] Jul 12 18:59:41.783374 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:41.783355 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-045df0-78f44744d8-s8dhh"] Jul 12 18:59:42.835282 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:42.835251 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" path="/var/lib/kubelet/pods/39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb/volumes" Jul 12 18:59:46.618136 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:46.618100 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:46.718370 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:46.718332 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:46.928407 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:46.928372 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:47.130562 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:47.130501 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:47.778174 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:47.778146 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 18:59:47.877836 ip-10-0-143-136 kubenswrapper[2568]: I0712 18:59:47.877805 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:01:00.510800 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:01:00.510762 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:01:00.611242 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:01:00.611213 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:02:47.656931 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:47.656896 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:02:53.178543 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.178494 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf"] Jul 12 19:02:53.179061 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.178779 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" podUID="4b2ef4d4-c870-40d3-974f-740f8ea0ce11" containerName="maas-api" containerID="cri-o://798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915" gracePeriod=30 Jul 12 19:02:53.208157 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.208132 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:02:53.380412 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.380380 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:02:53.426180 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.426157 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 19:02:53.560667 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.560570 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-openshift-service-ca\") pod \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " Jul 12 19:02:53.560667 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.560648 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsp2h\" (UniqueName: \"kubernetes.io/projected/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-kube-api-access-xsp2h\") pod \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " Jul 12 19:02:53.560900 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.560693 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-maas-api-tls\") pod \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\" (UID: \"4b2ef4d4-c870-40d3-974f-740f8ea0ce11\") " Jul 12 19:02:53.561012 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.560984 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b2ef4d4-c870-40d3-974f-740f8ea0ce11" (UID: "4b2ef4d4-c870-40d3-974f-740f8ea0ce11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 19:02:53.562736 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.562714 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-kube-api-access-xsp2h" (OuterVolumeSpecName: "kube-api-access-xsp2h") pod "4b2ef4d4-c870-40d3-974f-740f8ea0ce11" (UID: "4b2ef4d4-c870-40d3-974f-740f8ea0ce11"). InnerVolumeSpecName "kube-api-access-xsp2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:02:53.562807 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.562752 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b2ef4d4-c870-40d3-974f-740f8ea0ce11" (UID: "4b2ef4d4-c870-40d3-974f-740f8ea0ce11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 19:02:53.661876 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.661842 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsp2h\" (UniqueName: \"kubernetes.io/projected/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-kube-api-access-xsp2h\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:02:53.661876 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.661870 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:02:53.661876 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.661883 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b2ef4d4-c870-40d3-974f-740f8ea0ce11-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:02:53.670065 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.670039 2568 generic.go:356] "Generic (PLEG): container finished" podID="4b2ef4d4-c870-40d3-974f-740f8ea0ce11" containerID="798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915" exitCode=0 Jul 12 19:02:53.670191 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.670088 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" Jul 12 19:02:53.670191 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.670104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" event={"ID":"4b2ef4d4-c870-40d3-974f-740f8ea0ce11","Type":"ContainerDied","Data":"798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915"} Jul 12 19:02:53.670191 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.670163 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf" event={"ID":"4b2ef4d4-c870-40d3-974f-740f8ea0ce11","Type":"ContainerDied","Data":"193422ce64b373d93baa47fc596f03a0e07059dd9db25ed5ece0ab7c2c9267d9"} Jul 12 19:02:53.670291 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.670193 2568 scope.go:117] "RemoveContainer" containerID="798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915" Jul 12 19:02:53.678700 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.678683 2568 scope.go:117] "RemoveContainer" containerID="798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915" Jul 12 19:02:53.678931 ip-10-0-143-136 kubenswrapper[2568]: E0712 19:02:53.678911 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915\": container with ID starting with 798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915 not found: ID does not exist" containerID="798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915" Jul 12 19:02:53.678993 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.678939 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915"} err="failed to get container status \"798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915\": rpc error: code = NotFound desc = could not find container \"798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915\": container with ID starting with 798fbe551333ae781cba87f42173f6af0088f43960280b96056e2c615a310915 not found: ID does not exist" Jul 12 19:02:53.690583 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.690562 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf"] Jul 12 19:02:53.693929 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:53.693909 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-a249e9-d78d76fcd-cswnf"] Jul 12 19:02:54.841870 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:54.841828 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b2ef4d4-c870-40d3-974f-740f8ea0ce11" path="/var/lib/kubelet/pods/4b2ef4d4-c870-40d3-974f-740f8ea0ce11/volumes" Jul 12 19:02:56.687711 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:56.687682 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:02:56.784437 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:56.784398 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:02:58.815746 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:02:58.815716 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:00.045067 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:00.045032 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:00.404832 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:00.404770 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:03.295856 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.295822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:03.358425 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.358386 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689"] Jul 12 19:03:03.358772 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.358758 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" containerName="maas-api" Jul 12 19:03:03.358847 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.358774 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" containerName="maas-api" Jul 12 19:03:03.358847 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.358784 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b2ef4d4-c870-40d3-974f-740f8ea0ce11" containerName="maas-api" Jul 12 19:03:03.358847 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.358792 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b2ef4d4-c870-40d3-974f-740f8ea0ce11" containerName="maas-api" Jul 12 19:03:03.358942 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.358848 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="39ab1bbb-c8a4-49d6-be32-ac88c77ed9bb" containerName="maas-api" Jul 12 19:03:03.358942 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.358857 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b2ef4d4-c870-40d3-974f-740f8ea0ce11" containerName="maas-api" Jul 12 19:03:03.363245 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.363226 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.365791 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.365768 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-e6a009-serving-cert\"" Jul 12 19:03:03.373167 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.373142 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689"] Jul 12 19:03:03.374019 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.373994 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:03.444450 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.444360 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:03.453609 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.453583 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.453738 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.453625 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.453738 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.453653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkdjk\" (UniqueName: \"kubernetes.io/projected/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-kube-api-access-dkdjk\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.542145 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.542108 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:03.554292 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.554270 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.554405 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.554311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.554405 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.554339 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkdjk\" (UniqueName: \"kubernetes.io/projected/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-kube-api-access-dkdjk\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.554986 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.554961 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.556827 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.556800 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-maas-api-tls\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.565399 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.565369 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkdjk\" (UniqueName: \"kubernetes.io/projected/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-kube-api-access-dkdjk\") pod \"maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.674732 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.674695 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:03.811250 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.811219 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689"] Jul 12 19:03:03.813232 ip-10-0-143-136 kubenswrapper[2568]: W0712 19:03:03.813205 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c791ed4_aaa5_471e_b1d2_20b4d28268d9.slice/crio-397c95f5388959972749e093f8c9e0f90c4b27d8f24092fcddd3eb46181a2be4 WatchSource:0}: Error finding container 397c95f5388959972749e093f8c9e0f90c4b27d8f24092fcddd3eb46181a2be4: Status 404 returned error can't find the container with id 397c95f5388959972749e093f8c9e0f90c4b27d8f24092fcddd3eb46181a2be4 Jul 12 19:03:03.814759 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.814731 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 19:03:03.815308 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.815257 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:03.875176 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:03.875146 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:04.728899 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:04.728864 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" event={"ID":"3c791ed4-aaa5-471e-b1d2-20b4d28268d9","Type":"ContainerStarted","Data":"7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094"} Jul 12 19:03:04.729299 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:04.728920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" event={"ID":"3c791ed4-aaa5-471e-b1d2-20b4d28268d9","Type":"ContainerStarted","Data":"397c95f5388959972749e093f8c9e0f90c4b27d8f24092fcddd3eb46181a2be4"} Jul 12 19:03:04.729299 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:04.728957 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:04.747378 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:04.747333 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" podStartSLOduration=1.216085186 podStartE2EDuration="1.747317081s" totalImagesPullingTime="531.231895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:03:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:03:04.745619819 +0000 UTC m=+4178.389462545" watchObservedRunningTime="2026-07-12 19:03:04.747317081 +0000 UTC m=+4178.391159804" Jul 12 19:03:07.787896 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:07.787861 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:08.106871 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:08.106788 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:08.391459 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:08.391380 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:08.478913 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:08.478878 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:10.738522 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:10.738485 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:03:11.299214 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.299181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:11.370586 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.370555 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl"] Jul 12 19:03:11.370775 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.370745 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:11.374014 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.373994 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.376421 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.376399 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-53dad9-serving-cert\"" Jul 12 19:03:11.385187 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.385165 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl"] Jul 12 19:03:11.422103 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.422075 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84f774d2-2e8a-482d-96f5-b585d7a64a5b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.422282 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.422155 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.422282 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.422188 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf996\" (UniqueName: \"kubernetes.io/projected/84f774d2-2e8a-482d-96f5-b585d7a64a5b-kube-api-access-zf996\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.435582 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.435553 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:11.523461 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.523425 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.523461 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.523463 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf996\" (UniqueName: \"kubernetes.io/projected/84f774d2-2e8a-482d-96f5-b585d7a64a5b-kube-api-access-zf996\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.523740 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.523531 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84f774d2-2e8a-482d-96f5-b585d7a64a5b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.523740 ip-10-0-143-136 kubenswrapper[2568]: E0712 19:03:11.523600 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-serving-cert: secret "maas-api-e2e-shared-b-53dad9-serving-cert" not found Jul 12 19:03:11.523740 ip-10-0-143-136 kubenswrapper[2568]: E0712 19:03:11.523679 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls podName:84f774d2-2e8a-482d-96f5-b585d7a64a5b nodeName:}" failed. No retries permitted until 2026-07-12 19:03:12.023661981 +0000 UTC m=+4185.667504686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls") pod "maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" (UID: "84f774d2-2e8a-482d-96f5-b585d7a64a5b") : secret "maas-api-e2e-shared-b-53dad9-serving-cert" not found Jul 12 19:03:11.524171 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.524154 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84f774d2-2e8a-482d-96f5-b585d7a64a5b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.531448 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.531425 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf996\" (UniqueName: \"kubernetes.io/projected/84f774d2-2e8a-482d-96f5-b585d7a64a5b-kube-api-access-zf996\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:11.601789 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.601698 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:11.836097 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.836066 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:11.936578 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:11.936545 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:12.028064 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:12.028035 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:12.030466 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:12.030434 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls\") pod \"maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:12.288153 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:12.288069 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:12.430427 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:12.430403 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl"] Jul 12 19:03:12.431847 ip-10-0-143-136 kubenswrapper[2568]: W0712 19:03:12.431820 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f774d2_2e8a_482d_96f5_b585d7a64a5b.slice/crio-b5850c913f62dc736590c13478aa5c38990bfaf904659002effa2a567ce0137f WatchSource:0}: Error finding container b5850c913f62dc736590c13478aa5c38990bfaf904659002effa2a567ce0137f: Status 404 returned error can't find the container with id b5850c913f62dc736590c13478aa5c38990bfaf904659002effa2a567ce0137f Jul 12 19:03:12.766864 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:12.766826 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" event={"ID":"84f774d2-2e8a-482d-96f5-b585d7a64a5b","Type":"ContainerStarted","Data":"b5850c913f62dc736590c13478aa5c38990bfaf904659002effa2a567ce0137f"} Jul 12 19:03:13.775603 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:13.775571 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" event={"ID":"84f774d2-2e8a-482d-96f5-b585d7a64a5b","Type":"ContainerStarted","Data":"d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a"} Jul 12 19:03:13.776007 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:13.775667 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:13.792079 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:13.792028 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" podStartSLOduration=2.4110881969999998 podStartE2EDuration="2.792014552s" totalImagesPullingTime="380.926355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:03:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:03:13.790606149 +0000 UTC m=+4187.434448873" watchObservedRunningTime="2026-07-12 19:03:13.792014552 +0000 UTC m=+4187.435857327" Jul 12 19:03:15.792009 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:15.791973 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:19.786276 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:19.786250 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:03:25.130669 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:25.130637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:25.276470 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:25.276437 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:25.375852 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:25.375825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:32.567040 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:32.567009 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:32.664215 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:32.664184 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:03:46.224183 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:03:46.224152 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:05:10.390292 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:05:10.390257 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:05:10.491591 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:05:10.491558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:05:24.094152 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:05:24.094075 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:05:50.428214 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:05:50.428182 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:05:51.221280 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:05:51.221246 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:05:56.209216 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:05:56.209181 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:05:56.768498 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:05:56.768460 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:05:56.839494 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:05:56.839462 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:06:00.890090 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:06:00.890055 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:06:32.931941 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:06:32.931912 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:07:13.872778 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:13.872713 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:07:14.804830 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:14.804795 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:07:22.401405 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.401371 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx"] Jul 12 19:07:22.404461 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.404443 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.408486 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.408460 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"default-dockercfg-vgl9x\"" Jul 12 19:07:22.408486 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.408478 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"rate-test-model-e6a009-kserve-self-signed-certs\"" Jul 12 19:07:22.408702 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.408545 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"kube-root-ca.crt\"" Jul 12 19:07:22.408702 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.408474 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e6a009\"/\"openshift-service-ca.crt\"" Jul 12 19:07:22.411817 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.411796 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx"] Jul 12 19:07:22.498843 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.498812 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:07:22.511357 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.511324 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tmp-dir\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.511586 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.511396 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62n84\" (UniqueName: \"kubernetes.io/projected/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-kube-api-access-62n84\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.511586 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.511459 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-home\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.511586 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.511526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-dshm\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.511769 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.511632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-model-cache\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.511769 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.511656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tls-certs\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612153 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612123 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tmp-dir\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612360 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612167 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62n84\" (UniqueName: \"kubernetes.io/projected/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-kube-api-access-62n84\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612360 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-home\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612360 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612239 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-dshm\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612360 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612297 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-model-cache\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612360 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612321 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tls-certs\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612674 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tmp-dir\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612722 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612681 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-home\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.612766 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.612722 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-model-cache\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.615210 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.615183 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-dshm\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.615451 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.615429 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tls-certs\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.620329 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.620304 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62n84\" (UniqueName: \"kubernetes.io/projected/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-kube-api-access-62n84\") pod \"rate-test-model-e6a009-kserve-599fc5d5bc-fcspx\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.713935 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.713838 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:22.843151 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.843126 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx"] Jul 12 19:07:22.845438 ip-10-0-143-136 kubenswrapper[2568]: W0712 19:07:22.845402 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41f3a17c_fb49_4d1c_886b_49b3dc80bcc1.slice/crio-6e681efdc5fdbf29735f21afb93b559b409432080214c4ca67c94e59029f130c WatchSource:0}: Error finding container 6e681efdc5fdbf29735f21afb93b559b409432080214c4ca67c94e59029f130c: Status 404 returned error can't find the container with id 6e681efdc5fdbf29735f21afb93b559b409432080214c4ca67c94e59029f130c Jul 12 19:07:22.997191 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.997101 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" event={"ID":"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1","Type":"ContainerStarted","Data":"8dff8cfdf8ea994d8486224ef81fd690ce301d9e84472c6c7ce633d6ab85680d"} Jul 12 19:07:22.997191 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:22.997144 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" event={"ID":"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1","Type":"ContainerStarted","Data":"6e681efdc5fdbf29735f21afb93b559b409432080214c4ca67c94e59029f130c"} Jul 12 19:07:23.014956 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:23.014907 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" podStartSLOduration=1.014893696 podStartE2EDuration="1.014893696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 19:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:07:23.012372949 +0000 UTC m=+4436.656215698" watchObservedRunningTime="2026-07-12 19:07:23.014893696 +0000 UTC m=+4436.658736461" Jul 12 19:07:32.714636 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:32.714598 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:32.715084 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:32.714674 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:32.727338 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:32.727317 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:33.056552 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:33.056459 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:07:51.936300 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:51.936270 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:07:57.433922 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:57.433888 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:07:57.535456 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:57.535425 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:07:58.475407 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:58.475373 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:07:58.575674 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:07:58.575637 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:08:25.909795 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:25.909685 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:08:27.391041 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:27.391011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:08:29.216965 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:29.216930 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx"] Jul 12 19:08:29.235692 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:29.235662 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:08:30.120452 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:30.120409 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:08:33.201140 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:33.201111 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:08:33.299159 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:33.299127 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:08:44.237499 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.237438 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" podUID="41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" containerName="main" containerID="cri-o://8dff8cfdf8ea994d8486224ef81fd690ce301d9e84472c6c7ce633d6ab85680d" gracePeriod=45 Jul 12 19:08:44.395281 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.395242 2568 generic.go:356] "Generic (PLEG): container finished" podID="41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" containerID="8dff8cfdf8ea994d8486224ef81fd690ce301d9e84472c6c7ce633d6ab85680d" exitCode=0 Jul 12 19:08:44.395441 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.395309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" event={"ID":"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1","Type":"ContainerDied","Data":"8dff8cfdf8ea994d8486224ef81fd690ce301d9e84472c6c7ce633d6ab85680d"} Jul 12 19:08:44.492913 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.492852 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:08:44.615778 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.615744 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-dshm\") pod \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " Jul 12 19:08:44.615977 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.615820 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62n84\" (UniqueName: \"kubernetes.io/projected/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-kube-api-access-62n84\") pod \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " Jul 12 19:08:44.615977 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.615840 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tmp-dir\") pod \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " Jul 12 19:08:44.615977 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.615864 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-model-cache\") pod \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " Jul 12 19:08:44.615977 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.615913 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-home\") pod \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " Jul 12 19:08:44.615977 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.615941 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tls-certs\") pod \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\" (UID: \"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1\") " Jul 12 19:08:44.616285 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.616108 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" (UID: "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:08:44.616285 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.616195 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tmp-dir\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:08:44.616285 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.616167 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-model-cache" (OuterVolumeSpecName: "model-cache") pod "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" (UID: "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:08:44.616285 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.616228 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-home" (OuterVolumeSpecName: "home") pod "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" (UID: "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:08:44.618087 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.618057 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-dshm" (OuterVolumeSpecName: "dshm") pod "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" (UID: "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 19:08:44.618195 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.618111 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" (UID: "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 19:08:44.618195 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.618128 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-kube-api-access-62n84" (OuterVolumeSpecName: "kube-api-access-62n84") pod "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" (UID: "41f3a17c-fb49-4d1c-886b-49b3dc80bcc1"). InnerVolumeSpecName "kube-api-access-62n84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:08:44.717344 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.717309 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-dshm\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:08:44.717344 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.717336 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62n84\" (UniqueName: \"kubernetes.io/projected/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-kube-api-access-62n84\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:08:44.717344 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.717345 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-model-cache\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:08:44.717344 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.717354 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-home\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:08:44.717645 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:44.717363 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1-tls-certs\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:08:45.402690 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:45.402656 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" Jul 12 19:08:45.403174 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:45.402698 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx" event={"ID":"41f3a17c-fb49-4d1c-886b-49b3dc80bcc1","Type":"ContainerDied","Data":"6e681efdc5fdbf29735f21afb93b559b409432080214c4ca67c94e59029f130c"} Jul 12 19:08:45.403174 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:45.402756 2568 scope.go:117] "RemoveContainer" containerID="8dff8cfdf8ea994d8486224ef81fd690ce301d9e84472c6c7ce633d6ab85680d" Jul 12 19:08:45.418883 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:45.418859 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx"] Jul 12 19:08:45.422216 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:45.422195 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e6a009/rate-test-model-e6a009-kserve-599fc5d5bc-fcspx"] Jul 12 19:08:46.835384 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:08:46.835356 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" path="/var/lib/kubelet/pods/41f3a17c-fb49-4d1c-886b-49b3dc80bcc1/volumes" Jul 12 19:09:00.661826 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:09:00.661794 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:09:00.762747 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:09:00.762714 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:10:49.896717 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:10:49.896676 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:30.593820 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:30.593746 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:36.229625 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.229592 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689"] Jul 12 19:11:36.230116 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.229827 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" podUID="3c791ed4-aaa5-471e-b1d2-20b4d28268d9" containerName="maas-api" containerID="cri-o://7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094" gracePeriod=30 Jul 12 19:11:36.248897 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.248868 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:36.489364 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.489309 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:11:36.580380 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.580347 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:36.584804 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.584785 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-openshift-service-ca\") pod \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " Jul 12 19:11:36.584905 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.584818 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkdjk\" (UniqueName: \"kubernetes.io/projected/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-kube-api-access-dkdjk\") pod \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " Jul 12 19:11:36.584978 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.584913 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-maas-api-tls\") pod \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\" (UID: \"3c791ed4-aaa5-471e-b1d2-20b4d28268d9\") " Jul 12 19:11:36.585185 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.585161 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3c791ed4-aaa5-471e-b1d2-20b4d28268d9" (UID: "3c791ed4-aaa5-471e-b1d2-20b4d28268d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 19:11:36.586962 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.586938 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3c791ed4-aaa5-471e-b1d2-20b4d28268d9" (UID: "3c791ed4-aaa5-471e-b1d2-20b4d28268d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 19:11:36.587038 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.586984 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-kube-api-access-dkdjk" (OuterVolumeSpecName: "kube-api-access-dkdjk") pod "3c791ed4-aaa5-471e-b1d2-20b4d28268d9" (UID: "3c791ed4-aaa5-471e-b1d2-20b4d28268d9"). InnerVolumeSpecName "kube-api-access-dkdjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:11:36.686031 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.685986 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:11:36.686031 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.686028 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:11:36.686031 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:36.686038 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkdjk\" (UniqueName: \"kubernetes.io/projected/3c791ed4-aaa5-471e-b1d2-20b4d28268d9-kube-api-access-dkdjk\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:11:37.238907 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.238872 2568 generic.go:356] "Generic (PLEG): container finished" podID="3c791ed4-aaa5-471e-b1d2-20b4d28268d9" containerID="7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094" exitCode=0 Jul 12 19:11:37.239371 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.238912 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" Jul 12 19:11:37.239371 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.238949 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" event={"ID":"3c791ed4-aaa5-471e-b1d2-20b4d28268d9","Type":"ContainerDied","Data":"7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094"} Jul 12 19:11:37.239371 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.239002 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689" event={"ID":"3c791ed4-aaa5-471e-b1d2-20b4d28268d9","Type":"ContainerDied","Data":"397c95f5388959972749e093f8c9e0f90c4b27d8f24092fcddd3eb46181a2be4"} Jul 12 19:11:37.239371 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.239038 2568 scope.go:117] "RemoveContainer" containerID="7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094" Jul 12 19:11:37.247216 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.247200 2568 scope.go:117] "RemoveContainer" containerID="7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094" Jul 12 19:11:37.247462 ip-10-0-143-136 kubenswrapper[2568]: E0712 19:11:37.247444 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094\": container with ID starting with 7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094 not found: ID does not exist" containerID="7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094" Jul 12 19:11:37.247535 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.247471 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094"} err="failed to get container status \"7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094\": rpc error: code = NotFound desc = could not find container \"7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094\": container with ID starting with 7c690a93aaea8e46b6190cb046b9cb60ea13ca342d52cafc59e1f6170c150094 not found: ID does not exist" Jul 12 19:11:37.254369 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.254347 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689"] Jul 12 19:11:37.257981 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:37.257963 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-e6a009-7d5b57c565-qm689"] Jul 12 19:11:38.835411 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:38.835378 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c791ed4-aaa5-471e-b1d2-20b4d28268d9" path="/var/lib/kubelet/pods/3c791ed4-aaa5-471e-b1d2-20b4d28268d9/volumes" Jul 12 19:11:42.095142 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:42.095111 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:42.195706 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:42.195669 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:42.356006 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:42.355908 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:42.584998 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:42.584960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:43.135612 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:43.135574 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:11:43.234503 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:11:43.234463 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:12:58.085698 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:12:58.085604 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:12:58.182850 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:12:58.182818 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:13:00.945960 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:13:00.945920 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:13:01.047256 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:13:01.047218 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:13:29.226699 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:13:29.226664 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:13:30.361672 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:13:30.361640 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:14:43.069792 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:43.069756 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:14:48.627557 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:48.627519 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl"] Jul 12 19:14:48.628131 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:48.627793 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" podUID="84f774d2-2e8a-482d-96f5-b585d7a64a5b" containerName="maas-api" containerID="cri-o://d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a" gracePeriod=30 Jul 12 19:14:48.658351 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:48.658315 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:14:48.879128 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:48.879075 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:14:48.953919 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:48.953877 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:14:49.010894 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.010858 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls\") pod \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " Jul 12 19:14:49.011086 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.010918 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf996\" (UniqueName: \"kubernetes.io/projected/84f774d2-2e8a-482d-96f5-b585d7a64a5b-kube-api-access-zf996\") pod \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " Jul 12 19:14:49.011086 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.010992 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84f774d2-2e8a-482d-96f5-b585d7a64a5b-openshift-service-ca\") pod \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\" (UID: \"84f774d2-2e8a-482d-96f5-b585d7a64a5b\") " Jul 12 19:14:49.011373 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.011345 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84f774d2-2e8a-482d-96f5-b585d7a64a5b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "84f774d2-2e8a-482d-96f5-b585d7a64a5b" (UID: "84f774d2-2e8a-482d-96f5-b585d7a64a5b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 19:14:49.013066 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.013035 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "84f774d2-2e8a-482d-96f5-b585d7a64a5b" (UID: "84f774d2-2e8a-482d-96f5-b585d7a64a5b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 19:14:49.013066 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.013045 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84f774d2-2e8a-482d-96f5-b585d7a64a5b-kube-api-access-zf996" (OuterVolumeSpecName: "kube-api-access-zf996") pod "84f774d2-2e8a-482d-96f5-b585d7a64a5b" (UID: "84f774d2-2e8a-482d-96f5-b585d7a64a5b"). InnerVolumeSpecName "kube-api-access-zf996". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 19:14:49.111911 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.111861 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84f774d2-2e8a-482d-96f5-b585d7a64a5b-openshift-service-ca\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:14:49.111911 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.111904 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84f774d2-2e8a-482d-96f5-b585d7a64a5b-maas-api-tls\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:14:49.111911 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.111914 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf996\" (UniqueName: \"kubernetes.io/projected/84f774d2-2e8a-482d-96f5-b585d7a64a5b-kube-api-access-zf996\") on node \"ip-10-0-143-136.ec2.internal\" DevicePath \"\"" Jul 12 19:14:49.147604 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.147571 2568 generic.go:356] "Generic (PLEG): container finished" podID="84f774d2-2e8a-482d-96f5-b585d7a64a5b" containerID="d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a" exitCode=0 Jul 12 19:14:49.147748 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.147620 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" Jul 12 19:14:49.147748 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.147652 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" event={"ID":"84f774d2-2e8a-482d-96f5-b585d7a64a5b","Type":"ContainerDied","Data":"d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a"} Jul 12 19:14:49.147748 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.147703 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl" event={"ID":"84f774d2-2e8a-482d-96f5-b585d7a64a5b","Type":"ContainerDied","Data":"b5850c913f62dc736590c13478aa5c38990bfaf904659002effa2a567ce0137f"} Jul 12 19:14:49.147748 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.147725 2568 scope.go:117] "RemoveContainer" containerID="d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a" Jul 12 19:14:49.156923 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.156906 2568 scope.go:117] "RemoveContainer" containerID="d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a" Jul 12 19:14:49.157162 ip-10-0-143-136 kubenswrapper[2568]: E0712 19:14:49.157147 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a\": container with ID starting with d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a not found: ID does not exist" containerID="d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a" Jul 12 19:14:49.157224 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.157170 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a"} err="failed to get container status \"d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a\": rpc error: code = NotFound desc = could not find container \"d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a\": container with ID starting with d5d921dc613c68fce8c347e5892f8f5c84a3871eda333f8e45b478708ff8a45a not found: ID does not exist" Jul 12 19:14:49.167419 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.167396 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl"] Jul 12 19:14:49.173274 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:49.173254 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-53dad9-7cdbf648f5-nhcwl"] Jul 12 19:14:50.835033 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:50.835003 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84f774d2-2e8a-482d-96f5-b585d7a64a5b" path="/var/lib/kubelet/pods/84f774d2-2e8a-482d-96f5-b585d7a64a5b/volumes" Jul 12 19:14:54.568423 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:54.568385 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:14:54.630875 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:54.630838 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:14:54.899257 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:54.899223 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:14:55.087201 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:14:55.087170 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:15:01.103526 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:01.103474 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-rnps5_84be2600-f092-4624-a397-0192870543db/manager/0.log" Jul 12 19:15:01.109416 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:01.109394 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-rnps5_84be2600-f092-4624-a397-0192870543db/copy-manifests/0.log" Jul 12 19:15:01.449436 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:01.449411 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7c9bb8c854-glss8_46857c31-9c6e-4464-9fa5-431c969821ff/maas-api/0.log" Jul 12 19:15:01.560922 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:01.560892 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5hv74_b8c3c6e3-9f22-4e74-8d48-fe4cf84bd6b9/postgres/0.log" Jul 12 19:15:02.324895 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.324860 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg_24826af1-295c-41b4-b2e9-de36b2efb604/util/0.log" Jul 12 19:15:02.330644 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.330620 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg_24826af1-295c-41b4-b2e9-de36b2efb604/pull/0.log" Jul 12 19:15:02.336232 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.336216 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg_24826af1-295c-41b4-b2e9-de36b2efb604/extract/0.log" Jul 12 19:15:02.476464 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.476441 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t_07236e8e-d972-4846-97db-67643cc9a723/util/0.log" Jul 12 19:15:02.487094 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.487068 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t_07236e8e-d972-4846-97db-67643cc9a723/pull/0.log" Jul 12 19:15:02.498145 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.498122 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t_07236e8e-d972-4846-97db-67643cc9a723/extract/0.log" Jul 12 19:15:02.604567 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.604472 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb_b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe/util/0.log" Jul 12 19:15:02.609674 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.609643 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb_b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe/pull/0.log" Jul 12 19:15:02.614622 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.614600 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb_b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe/extract/0.log" Jul 12 19:15:02.724751 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.724723 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh_f7fecf65-edda-4a2c-8050-79823a80ca2b/util/0.log" Jul 12 19:15:02.730332 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.730312 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh_f7fecf65-edda-4a2c-8050-79823a80ca2b/pull/0.log" Jul 12 19:15:02.735873 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:02.735853 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh_f7fecf65-edda-4a2c-8050-79823a80ca2b/extract/0.log" Jul 12 19:15:04.117684 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:04.117656 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7jm6s_953be662-d153-46f3-ab5f-375a5eba7a2f/discovery/0.log" Jul 12 19:15:04.567367 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:04.567336 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-6rbt5_41ee9192-b172-4087-8abd-78fc3880e40b/payload-pre-processing/0.log" Jul 12 19:15:04.791672 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:04.791644 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79486c5bd6-wvmlj_2fa32b08-267f-4092-b626-c4b0908a4d42/router/0.log" Jul 12 19:15:05.237972 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:05.237940 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-d4x7b_09f894d5-01ec-4f2a-a430-b4ba1268b4c8/main/0.log" Jul 12 19:15:05.350575 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:05.350550 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-t2w76_8ba97051-d449-4a1c-9f5a-59698d1f833d/main/0.log" Jul 12 19:15:05.572730 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:05.568572 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-24cfz_8d5ee50f-2d9f-42cb-8465-d2ddd40a3dbf/main/0.log" Jul 12 19:15:09.892231 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:09.892201 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:15:09.994340 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:09.994307 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" limit=1024 Jul 12 19:15:12.725292 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:12.725256 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-djndg_d907cd33-4123-4333-a38a-ac8e60b5ef04/global-pull-secret-syncer/0.log" Jul 12 19:15:12.830309 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:12.830275 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dnw88_b3322fe7-5ad0-4f6d-9df7-300b067e2082/konnectivity-agent/0.log" Jul 12 19:15:12.976224 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:12.976143 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-136.ec2.internal_73fc1951afb982df9c53cf26d8ff4718/haproxy/0.log" Jul 12 19:15:17.543222 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.543190 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg_24826af1-295c-41b4-b2e9-de36b2efb604/extract/0.log" Jul 12 19:15:17.571019 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.570984 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg_24826af1-295c-41b4-b2e9-de36b2efb604/util/0.log" Jul 12 19:15:17.591657 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.591633 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fh2jg_24826af1-295c-41b4-b2e9-de36b2efb604/pull/0.log" Jul 12 19:15:17.618831 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.618800 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t_07236e8e-d972-4846-97db-67643cc9a723/extract/0.log" Jul 12 19:15:17.638855 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.638833 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t_07236e8e-d972-4846-97db-67643cc9a723/util/0.log" Jul 12 19:15:17.662811 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.662790 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0bv55t_07236e8e-d972-4846-97db-67643cc9a723/pull/0.log" Jul 12 19:15:17.684711 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.684689 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb_b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe/extract/0.log" Jul 12 19:15:17.704632 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.704613 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb_b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe/util/0.log" Jul 12 19:15:17.724592 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.724567 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73btwvb_b15ab6d0-1584-46ec-ba13-9c85ca2aa9fe/pull/0.log" Jul 12 19:15:17.750157 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.750128 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh_f7fecf65-edda-4a2c-8050-79823a80ca2b/extract/0.log" Jul 12 19:15:17.771098 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.771078 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh_f7fecf65-edda-4a2c-8050-79823a80ca2b/util/0.log" Jul 12 19:15:17.790791 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:17.790771 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xd7hh_f7fecf65-edda-4a2c-8050-79823a80ca2b/pull/0.log" Jul 12 19:15:20.536641 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:20.536609 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-zpnbm_521e82e9-be97-48e8-98cc-912f83b22c23/cluster-monitoring-operator/0.log" Jul 12 19:15:20.714584 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:20.714558 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68wgp_2bfef6e6-74f1-41e6-acb2-fb369c86b753/node-exporter/0.log" Jul 12 19:15:20.739020 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:20.738993 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68wgp_2bfef6e6-74f1-41e6-acb2-fb369c86b753/kube-rbac-proxy/0.log" Jul 12 19:15:20.758857 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:20.758831 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-68wgp_2bfef6e6-74f1-41e6-acb2-fb369c86b753/init-textfile/0.log" Jul 12 19:15:21.233986 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:21.233954 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-gtlc5_aa2ec6df-14a1-4f35-82ed-2fbd6709f8dd/prometheus-operator-admission-webhook/0.log" Jul 12 19:15:22.897705 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.897670 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7"] Jul 12 19:15:22.898076 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.897976 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84f774d2-2e8a-482d-96f5-b585d7a64a5b" containerName="maas-api" Jul 12 19:15:22.898076 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.897986 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="84f774d2-2e8a-482d-96f5-b585d7a64a5b" containerName="maas-api" Jul 12 19:15:22.898076 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.897996 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c791ed4-aaa5-471e-b1d2-20b4d28268d9" containerName="maas-api" Jul 12 19:15:22.898076 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.898001 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c791ed4-aaa5-471e-b1d2-20b4d28268d9" containerName="maas-api" Jul 12 19:15:22.898076 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.898010 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" containerName="main" Jul 12 19:15:22.898076 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.898015 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" containerName="main" Jul 12 19:15:22.898076 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.898077 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c791ed4-aaa5-471e-b1d2-20b4d28268d9" containerName="maas-api" Jul 12 19:15:22.898295 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.898084 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="84f774d2-2e8a-482d-96f5-b585d7a64a5b" containerName="maas-api" Jul 12 19:15:22.898295 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.898091 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="41f3a17c-fb49-4d1c-886b-49b3dc80bcc1" containerName="main" Jul 12 19:15:22.901192 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.901171 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:22.903632 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.903611 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h56g7\"/\"kube-root-ca.crt\"" Jul 12 19:15:22.904319 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.904301 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-h56g7\"/\"openshift-service-ca.crt\"" Jul 12 19:15:22.904410 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.904307 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-h56g7\"/\"default-dockercfg-p4lkn\"" Jul 12 19:15:22.908704 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.908125 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7"] Jul 12 19:15:22.981265 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.981221 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-proc\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:22.981265 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.981265 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-podres\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:22.981479 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.981299 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6mt4\" (UniqueName: \"kubernetes.io/projected/f1798638-2ebf-411f-a893-6ff5a3f53e3b-kube-api-access-l6mt4\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:22.981479 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.981335 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-sys\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:22.981479 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:22.981392 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-lib-modules\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082255 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082209 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6mt4\" (UniqueName: \"kubernetes.io/projected/f1798638-2ebf-411f-a893-6ff5a3f53e3b-kube-api-access-l6mt4\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082427 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082309 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-sys\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082427 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-lib-modules\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082427 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082405 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-proc\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082611 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082431 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-podres\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082611 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082429 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-sys\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082611 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082495 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-proc\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082739 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082614 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-podres\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.082739 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.082631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1798638-2ebf-411f-a893-6ff5a3f53e3b-lib-modules\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.089762 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.089734 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6mt4\" (UniqueName: \"kubernetes.io/projected/f1798638-2ebf-411f-a893-6ff5a3f53e3b-kube-api-access-l6mt4\") pod \"perf-node-gather-daemonset-jd9x7\" (UID: \"f1798638-2ebf-411f-a893-6ff5a3f53e3b\") " pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.212328 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.212228 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:23.340912 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.340847 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7"] Jul 12 19:15:23.343766 ip-10-0-143-136 kubenswrapper[2568]: W0712 19:15:23.343733 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf1798638_2ebf_411f_a893_6ff5a3f53e3b.slice/crio-05b00a10e4434207c79ed885de8b6abaa54e75c6059d0ac48f56db391f418b95 WatchSource:0}: Error finding container 05b00a10e4434207c79ed885de8b6abaa54e75c6059d0ac48f56db391f418b95: Status 404 returned error can't find the container with id 05b00a10e4434207c79ed885de8b6abaa54e75c6059d0ac48f56db391f418b95 Jul 12 19:15:23.859119 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:23.859084 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-sns9z_749f64c4-70ba-42ac-aa63-ba470c87b5a5/download-server/0.log" Jul 12 19:15:24.317812 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:24.317776 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" event={"ID":"f1798638-2ebf-411f-a893-6ff5a3f53e3b","Type":"ContainerStarted","Data":"63395d1a0c2786ed71be38e129548727b00bc1e257b5e2ae61e1588021add677"} Jul 12 19:15:24.318180 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:24.317825 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" event={"ID":"f1798638-2ebf-411f-a893-6ff5a3f53e3b","Type":"ContainerStarted","Data":"05b00a10e4434207c79ed885de8b6abaa54e75c6059d0ac48f56db391f418b95"} Jul 12 19:15:24.318180 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:24.317919 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:24.334963 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:24.334914 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" podStartSLOduration=2.334897072 podStartE2EDuration="2.334897072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 19:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 19:15:24.333068934 +0000 UTC m=+4917.976911657" watchObservedRunningTime="2026-07-12 19:15:24.334897072 +0000 UTC m=+4917.978739795" Jul 12 19:15:25.453402 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:25.453372 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzstl_cee3c823-d6f7-4498-b550-bd04f28d6152/dns/0.log" Jul 12 19:15:25.473454 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:25.473419 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nzstl_cee3c823-d6f7-4498-b550-bd04f28d6152/kube-rbac-proxy/0.log" Jul 12 19:15:25.614062 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:25.614036 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-d7cld_bcaa496d-87e7-4d2f-a1de-2a67abf5cb68/dns-node-resolver/0.log" Jul 12 19:15:26.201792 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:26.201767 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-498ks_d1dec3aa-5c23-4a97-8827-90a57806b2b2/node-ca/0.log" Jul 12 19:15:27.409432 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:27.409402 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7jm6s_953be662-d153-46f3-ab5f-375a5eba7a2f/discovery/0.log" Jul 12 19:15:27.562869 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:27.562841 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-6rbt5_41ee9192-b172-4087-8abd-78fc3880e40b/payload-pre-processing/0.log" Jul 12 19:15:27.614068 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:27.613979 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79486c5bd6-wvmlj_2fa32b08-267f-4092-b626-c4b0908a4d42/router/0.log" Jul 12 19:15:28.315843 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:28.315811 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jvjpf_68db3a53-54e6-4a52-9b0b-63f9aedb0c83/serve-healthcheck-canary/0.log" Jul 12 19:15:28.791972 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:28.791930 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-qtd2t_74c2ade1-1a3b-4385-896c-daa000f2538b/insights-operator/0.log" Jul 12 19:15:28.876808 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:28.876782 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bjl5k_f8148f4a-8ab9-42c9-822e-8ba38b9274ed/kube-rbac-proxy/0.log" Jul 12 19:15:28.893825 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:28.893794 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bjl5k_f8148f4a-8ab9-42c9-822e-8ba38b9274ed/exporter/0.log" Jul 12 19:15:28.913335 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:28.913305 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bjl5k_f8148f4a-8ab9-42c9-822e-8ba38b9274ed/extractor/0.log" Jul 12 19:15:30.332961 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:30.332934 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-h56g7/perf-node-gather-daemonset-jd9x7" Jul 12 19:15:31.625445 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:31.625417 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-rnps5_84be2600-f092-4624-a397-0192870543db/manager/0.log" Jul 12 19:15:31.641177 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:31.641154 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5dbd4cc77c-rnps5_84be2600-f092-4624-a397-0192870543db/copy-manifests/0.log" Jul 12 19:15:33.043408 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:33.043319 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5484cf687f-dl8jk_3aadf229-7793-435d-a708-451cb010f705/manager/0.log" Jul 12 19:15:33.448731 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:33.448697 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7jm6s_953be662-d153-46f3-ab5f-375a5eba7a2f/discovery/0.log" Jul 12 19:15:33.553144 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:33.553115 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-6rbt5_41ee9192-b172-4087-8abd-78fc3880e40b/payload-pre-processing/0.log" Jul 12 19:15:33.584045 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:33.584020 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79486c5bd6-wvmlj_2fa32b08-267f-4092-b626-c4b0908a4d42/router/0.log" Jul 12 19:15:34.993347 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:34.993304 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-7jm6s_953be662-d153-46f3-ab5f-375a5eba7a2f/discovery/0.log" Jul 12 19:15:35.114688 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:35.114662 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-6rbt5_41ee9192-b172-4087-8abd-78fc3880e40b/payload-pre-processing/0.log" Jul 12 19:15:35.153361 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:35.153331 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79486c5bd6-wvmlj_2fa32b08-267f-4092-b626-c4b0908a4d42/router/0.log" Jul 12 19:15:37.995697 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:37.995664 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-q6v94_d6c13874-8138-412d-a1ca-ad2b1c9e6440/migrator/0.log" Jul 12 19:15:38.014349 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:38.014325 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-q6v94_d6c13874-8138-412d-a1ca-ad2b1c9e6440/graceful-termination/0.log" Jul 12 19:15:39.674145 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.674120 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxmk8_950b9259-0b9e-4847-838d-26aad602908c/kube-multus-additional-cni-plugins/0.log" Jul 12 19:15:39.694249 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.694223 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxmk8_950b9259-0b9e-4847-838d-26aad602908c/egress-router-binary-copy/0.log" Jul 12 19:15:39.713223 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.713198 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxmk8_950b9259-0b9e-4847-838d-26aad602908c/cni-plugins/0.log" Jul 12 19:15:39.736073 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.736046 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxmk8_950b9259-0b9e-4847-838d-26aad602908c/bond-cni-plugin/0.log" Jul 12 19:15:39.755348 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.755326 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxmk8_950b9259-0b9e-4847-838d-26aad602908c/routeoverride-cni/0.log" Jul 12 19:15:39.776159 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.776131 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxmk8_950b9259-0b9e-4847-838d-26aad602908c/whereabouts-cni-bincopy/0.log" Jul 12 19:15:39.795901 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.795879 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wxmk8_950b9259-0b9e-4847-838d-26aad602908c/whereabouts-cni/0.log" Jul 12 19:15:39.870020 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.869993 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ww5j4_1b709f4a-171e-49cd-941f-f6856018b6a2/kube-multus/0.log" Jul 12 19:15:39.931566 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.931464 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nxhsg_d6ee4bc0-6991-412a-884c-dbc5e5ef7665/network-metrics-daemon/0.log" Jul 12 19:15:39.948109 ip-10-0-143-136 kubenswrapper[2568]: I0712 19:15:39.948080 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nxhsg_d6ee4bc0-6991-412a-884c-dbc5e5ef7665/kube-rbac-proxy/0.log"