Jul 10 17:36:51.220860 ip-10-0-131-229 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 17:36:51.220872 ip-10-0-131-229 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 17:36:51.220879 ip-10-0-131-229 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 17:36:51.221093 ip-10-0-131-229 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 17:37:01.280561 ip-10-0-131-229 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 17:37:01.280585 ip-10-0-131-229 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b9a5b6548aea4d72a79428987f646c0b -- Jul 10 17:38:42.037252 ip-10-0-131-229 systemd[1]: Starting Kubernetes Kubelet... Jul 10 17:38:42.489243 ip-10-0-131-229 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 17:38:42.489243 ip-10-0-131-229 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 17:38:42.489243 ip-10-0-131-229 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 17:38:42.489243 ip-10-0-131-229 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 17:38:42.489243 ip-10-0-131-229 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 17:38:42.491354 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.491236 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 17:38:42.496095 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496080 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 17:38:42.496095 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496095 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496099 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496102 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496105 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496109 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496112 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496115 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496117 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496120 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496124 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496128 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496131 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496134 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496137 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496140 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496144 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496147 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496155 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496158 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 17:38:42.496159 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496161 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496165 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496167 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496170 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496174 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496176 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496179 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496182 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496186 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496190 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496193 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496195 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496198 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496200 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496203 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496206 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496208 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496210 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496213 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 17:38:42.496624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496215 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496218 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496221 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496223 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496226 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496229 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496231 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496233 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496236 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496238 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496240 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496244 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496246 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496249 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496251 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496254 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496257 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496259 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496262 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496264 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 17:38:42.497106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496266 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496269 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496271 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496273 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496276 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496278 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496280 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496283 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496285 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496288 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496290 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496292 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496295 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496298 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496300 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496303 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496305 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496307 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496309 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496312 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 17:38:42.497624 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496314 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496316 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496319 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496321 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496324 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496327 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496329 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496332 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496334 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496337 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496339 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496342 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496344 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496347 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496349 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496351 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496354 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496357 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496359 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.496362 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 17:38:42.498123 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497078 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497086 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497089 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497091 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497094 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497105 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497108 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497111 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497113 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497116 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497119 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497121 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497124 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497128 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497132 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497135 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497138 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497140 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497143 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 17:38:42.498595 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497147 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497150 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497152 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497155 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497158 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497160 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497163 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497165 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497168 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497170 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497173 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497175 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497177 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497180 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497182 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497184 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497187 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497189 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497192 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497195 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 17:38:42.499071 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497197 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497200 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497202 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497205 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497207 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497210 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497213 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497216 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497218 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497221 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497223 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497225 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497228 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497231 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497234 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497236 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497238 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497241 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497244 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497247 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 17:38:42.499558 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497250 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497252 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497254 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497257 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497259 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497263 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497267 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497270 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497274 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497278 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497281 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497283 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497289 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497292 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497295 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497298 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497300 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497303 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497305 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 17:38:42.500114 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497308 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497310 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497313 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497316 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497318 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497321 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497323 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497325 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497328 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497330 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497332 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497335 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497337 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497339 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497342 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497344 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497347 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497349 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497351 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497354 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 17:38:42.500576 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497356 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497713 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497718 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497721 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497724 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497727 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497730 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497732 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497735 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497737 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497740 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497743 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497746 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497748 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497751 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497753 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497756 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497759 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497762 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497764 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 17:38:42.501085 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497767 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497769 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497772 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497774 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497777 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497779 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497782 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497784 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497787 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497789 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497791 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497794 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497796 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497799 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497801 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497804 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497806 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497809 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497811 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 17:38:42.501566 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497814 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497816 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497819 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497823 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497825 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497828 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497830 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497833 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497835 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497838 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497840 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497843 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497845 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497847 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497850 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497854 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497857 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497859 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497862 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497865 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 17:38:42.502051 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497868 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497870 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497873 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497875 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497878 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497880 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497883 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497885 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497888 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497890 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497892 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497895 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497898 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497900 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497902 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497907 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497909 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497913 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497915 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 17:38:42.502523 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497918 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497922 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497924 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497926 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497929 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497931 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497933 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497936 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497938 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497941 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497943 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497947 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497950 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497953 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497955 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497958 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497960 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497962 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497982 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497986 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 17:38:42.502984 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497990 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.497993 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498054 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498061 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498066 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498070 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498080 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498084 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498089 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498093 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498096 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498100 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498103 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498106 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498109 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498112 2568 flags.go:64] FLAG: --cgroup-root="" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498115 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498118 2568 flags.go:64] FLAG: --client-ca-file="" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498120 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498123 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498128 2568 flags.go:64] FLAG: --cluster-domain="" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498131 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498134 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498137 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 17:38:42.503460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498141 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498144 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498147 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498150 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498153 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498155 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498158 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498161 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498164 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498167 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498170 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498173 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498176 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498178 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498182 2568 flags.go:64] FLAG: --enable-server="true" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498185 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498193 2568 flags.go:64] FLAG: --event-burst="100" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498196 2568 flags.go:64] FLAG: --event-qps="50" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498198 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498201 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498204 2568 flags.go:64] FLAG: --eviction-hard="" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498207 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498210 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498213 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498216 2568 flags.go:64] FLAG: --eviction-soft="" Jul 10 17:38:42.504063 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498219 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498222 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498225 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498227 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498230 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498233 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498236 2568 flags.go:64] FLAG: --feature-gates="" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498239 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498242 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498244 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498247 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498250 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498253 2568 flags.go:64] FLAG: --help="false" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498256 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498259 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498261 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498264 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498267 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498270 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498273 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498276 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498281 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498284 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 17:38:42.504655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498288 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498291 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498294 2568 flags.go:64] FLAG: --kube-reserved="" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498297 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498300 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498302 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498305 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498307 2568 flags.go:64] FLAG: --lock-file="" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498310 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498313 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498316 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498321 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498323 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498326 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498329 2568 flags.go:64] FLAG: --logging-format="text" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498331 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498334 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498337 2568 flags.go:64] FLAG: --manifest-url="" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498340 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498344 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498346 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498350 2568 flags.go:64] FLAG: --max-pods="110" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498353 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498356 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498358 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 17:38:42.505355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498361 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498364 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498366 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498369 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498375 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498379 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498382 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498386 2568 flags.go:64] FLAG: --pod-cidr="" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498389 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498394 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498397 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498400 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498403 2568 flags.go:64] FLAG: --port="10250" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498406 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498409 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04ef4bd002af4a17b" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498412 2568 flags.go:64] FLAG: --qos-reserved="" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498415 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498418 2568 flags.go:64] FLAG: --register-node="true" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498420 2568 flags.go:64] FLAG: --register-with-taints="" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498429 2568 flags.go:64] FLAG: --registry-burst="10" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498431 2568 flags.go:64] FLAG: --registry-qps="5" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498434 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498437 2568 flags.go:64] FLAG: --reserved-memory="" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498440 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498443 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 17:38:42.506068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498446 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498448 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498451 2568 flags.go:64] FLAG: --runonce="false" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498454 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498457 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498459 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498462 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498465 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498467 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498470 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498473 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498475 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498478 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498482 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498486 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498489 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498492 2568 flags.go:64] FLAG: --system-cgroups="" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498495 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498500 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498502 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498505 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498509 2568 flags.go:64] FLAG: --tls-min-version="" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498512 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498515 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498517 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 17:38:42.506659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498520 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498523 2568 flags.go:64] FLAG: --v="2" Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498526 2568 flags.go:64] FLAG: --version="false" Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498530 2568 flags.go:64] FLAG: --vmodule="" Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498533 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.498536 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498643 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498647 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498650 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498653 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498656 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498660 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498663 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498665 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498668 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498670 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498673 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498676 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498678 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498681 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498685 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 17:38:42.507325 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498688 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498692 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498696 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498699 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498702 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498705 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498708 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498710 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498713 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498715 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498718 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498721 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498723 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498725 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498728 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498731 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498735 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.498738 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506864 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506871 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 17:38:42.507830 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506876 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506880 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506884 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506888 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506892 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506896 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506900 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506903 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506906 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506909 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506913 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506916 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506920 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506924 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506929 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506933 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506940 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506943 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506948 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 17:38:42.508331 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506952 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506957 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506962 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506983 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506988 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506992 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506997 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.506999 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507002 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507004 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507007 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507009 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507012 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507014 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507017 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507019 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507022 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507024 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507026 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507029 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 17:38:42.508784 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507031 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507034 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507036 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507038 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507041 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507044 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507046 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507049 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507051 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507054 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507057 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507060 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507062 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507064 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507066 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507069 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507072 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507074 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507076 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507079 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 17:38:42.509337 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507081 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 17:38:42.509826 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507084 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 17:38:42.509826 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507086 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 17:38:42.509826 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507090 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 17:38:42.509826 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.507093 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 17:38:42.509826 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.507878 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 17:38:42.515536 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.515517 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 17:38:42.515536 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.515536 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 17:38:42.516432 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.516423 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 17:38:42.516463 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.516434 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 17:38:42.516509 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516501 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 17:38:42.516509 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516509 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516513 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516517 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516519 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516522 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516524 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516527 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516530 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516533 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516535 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516538 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516546 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516549 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516552 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516554 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516556 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516559 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516561 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516564 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 17:38:42.516567 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516566 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516569 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516571 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516574 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516579 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516582 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516585 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516588 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516591 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516594 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516596 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516598 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516601 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516604 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516606 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516608 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516611 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516613 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516616 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516618 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 17:38:42.517060 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516620 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516623 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516625 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516627 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516630 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516637 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516640 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516643 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516645 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516647 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516650 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516652 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516654 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516657 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516660 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516669 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516671 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516674 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516676 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516679 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 17:38:42.517536 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516681 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516683 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516686 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516689 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516691 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516694 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516696 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516698 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516701 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516703 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516705 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516708 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516711 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516715 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516718 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516720 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516723 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516725 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516733 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516736 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 17:38:42.518152 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516738 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516741 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516743 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516746 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516749 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516751 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516754 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516756 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516759 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516761 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516764 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516766 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516768 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516772 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516775 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516778 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516780 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516783 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 17:38:42.518623 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516786 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.516791 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516951 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516956 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516959 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516962 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516978 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516981 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516984 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516986 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516988 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516991 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.516993 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517001 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517004 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 17:38:42.519106 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517007 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517009 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517011 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517014 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517020 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517022 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517025 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517028 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517030 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517033 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517035 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517038 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517040 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517042 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517045 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517047 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517050 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517052 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517055 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517057 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 17:38:42.519467 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517060 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517062 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517064 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517067 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517070 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517072 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517075 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517077 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517081 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517084 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517088 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517098 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517101 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517103 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517106 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517108 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517112 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517114 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517117 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 17:38:42.519941 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517119 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517121 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517125 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517128 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517131 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517134 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517136 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517139 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517141 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517144 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517146 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517149 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517151 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517153 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517156 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517158 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517162 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517165 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517167 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 17:38:42.520420 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517170 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517172 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517175 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517178 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517180 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517183 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517191 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517193 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517196 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517198 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517201 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517204 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517206 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517209 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517211 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517213 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517216 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517218 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517221 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517223 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 17:38:42.520875 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517226 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517228 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517230 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517233 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517235 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517237 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517240 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:42.517242 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.517247 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.517348 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 17:38:42.521385 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.520150 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 17:38:42.521788 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.521775 2568 server.go:1019] "Starting client certificate rotation" Jul 10 17:38:42.521927 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.521911 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 17:38:42.521991 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.521977 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 17:38:42.524613 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.524600 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 17:38:42.531640 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.531619 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 17:38:42.545126 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.545105 2568 log.go:25] "Validated CRI v1 runtime API" Jul 10 17:38:42.548521 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.548498 2568 log.go:25] "Validated CRI v1 image API" Jul 10 17:38:42.549485 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.549469 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 17:38:42.551946 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.551923 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 17:38:42.552691 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.552674 2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c1160efd-1069-407f-92bb-ca4f2e68a7c8:/dev/nvme0n1p4 e4d48d8f-852d-4bdd-8400-03f02102f908:/dev/nvme0n1p3] Jul 10 17:38:42.552750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.552691 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 17:38:42.558513 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.558409 2568 manager.go:216] Machine: {Timestamp:2026-07-10 17:38:42.557004663 +0000 UTC m=+0.384653422 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100501 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28467aba4594f6f9fd2d99835de1a7 SystemUUID:ec28467a-ba45-94f6-f9fd-2d99835de1a7 BootID:b9a5b654-8aea-4d72-a794-28987f646c0b Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:01:d4:1e:d5:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:01:d4:1e:d5:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:9c:af:13:8b:6e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 17:38:42.558513 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.558509 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 17:38:42.558611 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.558577 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 17:38:42.562257 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.562232 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 17:38:42.562395 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.562260 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-229.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 17:38:42.563088 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.563077 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 17:38:42.563118 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.563090 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 17:38:42.563118 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.563102 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 17:38:42.563978 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.563956 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 17:38:42.565873 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.565862 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 17:38:42.571335 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571323 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 10 17:38:42.571449 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571440 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 17:38:42.571478 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571474 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 10 17:38:42.571507 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571489 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 17:38:42.571507 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571502 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 17:38:42.571562 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571512 2568 kubelet.go:398] "Adding apiserver pod source" Jul 10 17:38:42.571562 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571520 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 17:38:42.571866 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571855 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 17:38:42.571905 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.571875 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 17:38:42.574295 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.574261 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 17:38:42.576613 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.576597 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 17:38:42.576691 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.576629 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 17:38:42.576815 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.576801 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 17:38:42.576845 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.576833 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 17:38:42.576845 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.576844 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 17:38:42.576901 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.576855 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 17:38:42.576901 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.576871 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 17:38:42.576901 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.576882 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 17:38:42.577423 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.577412 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 17:38:42.577452 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.577423 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 17:38:42.577452 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.577435 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 17:38:42.577452 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.577441 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 17:38:42.577452 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.577450 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 17:38:42.577564 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.577458 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 17:38:42.579207 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.579192 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 17:38:42.579253 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.579208 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 17:38:42.582653 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.582639 2568 server.go:1268] "Started kubelet" Jul 10 17:38:42.583156 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.583044 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 17:38:42.583571 ip-10-0-131-229 systemd[1]: Started Kubernetes Kubelet. Jul 10 17:38:42.587112 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.583059 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 17:38:42.587206 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.587171 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 17:38:42.587956 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.587929 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-229.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 17:38:42.588053 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.587936 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 17:38:42.588103 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.588086 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 17:38:42.588688 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.588648 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 17:38:42.589958 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.589917 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 17:38:42.589958 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.589943 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 10 17:38:42.593003 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.591847 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-229.ec2.internal.18c0fe5bc794c3e1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-229.ec2.internal,UID:ip-10-0-131-229.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-229.ec2.internal,},FirstTimestamp:2026-07-10 17:38:42.582610913 +0000 UTC m=+0.410259671,LastTimestamp:2026-07-10 17:38:42.582610913 +0000 UTC m=+0.410259671,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-229.ec2.internal,}" Jul 10 17:38:42.596790 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.596766 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 17:38:42.596790 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.596773 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 17:38:42.596932 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.596859 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:42.597595 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.597576 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 17:38:42.597595 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.597597 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 17:38:42.597751 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.597735 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 10 17:38:42.597751 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.597751 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 10 17:38:42.597865 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.597764 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 17:38:42.598037 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.598019 2568 factory.go:55] Registering systemd factory Jul 10 17:38:42.598103 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.598041 2568 factory.go:223] Registration of the systemd container factory successfully Jul 10 17:38:42.598267 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.598253 2568 factory.go:153] Registering CRI-O factory Jul 10 17:38:42.598333 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.598271 2568 factory.go:223] Registration of the crio container factory successfully Jul 10 17:38:42.598333 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.598318 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 17:38:42.598403 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.598348 2568 factory.go:103] Registering Raw factory Jul 10 17:38:42.598403 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.598366 2568 manager.go:1196] Started watching for new ooms in manager Jul 10 17:38:42.599059 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.599043 2568 manager.go:349] Starting recovery of all containers Jul 10 17:38:42.600166 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.600140 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q757m" Jul 10 17:38:42.605562 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.605416 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 17:38:42.607517 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.607492 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q757m" Jul 10 17:38:42.609071 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.609049 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-229.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 17:38:42.609166 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.609137 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 17:38:42.609396 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.609378 2568 manager.go:354] Recovery completed Jul 10 17:38:42.613799 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.613786 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 17:38:42.615958 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.615942 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 17:38:42.616047 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.615985 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 17:38:42.616047 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.615997 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 17:38:42.616516 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.616500 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 17:38:42.616516 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.616514 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 17:38:42.616612 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.616516 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 17:38:42.616612 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.616530 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 10 17:38:42.618391 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.618367 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 17:38:42.618466 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.618397 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 17:38:42.618466 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.618413 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 17:38:42.619165 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.619151 2568 policy_none.go:49] "None policy: Start" Jul 10 17:38:42.619228 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.619168 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 17:38:42.619228 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.619179 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 17:38:42.621205 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.621195 2568 policy_none.go:47] "Start" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.662607 2568 manager.go:337] "Starting Device Plugin manager" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.662639 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.662652 2568 server.go:87] "Starting device plugin registration server" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.662863 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.662877 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.662951 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.663059 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.663069 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.663517 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 17:38:42.683096 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.663566 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:42.749691 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.749636 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 17:38:42.749691 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.749680 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 17:38:42.749821 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.749701 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 17:38:42.749821 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.749737 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 17:38:42.751999 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.751981 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 17:38:42.763925 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.763910 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 17:38:42.764809 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.764796 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 17:38:42.764866 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.764824 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 17:38:42.764866 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.764835 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 17:38:42.764866 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.764856 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.775666 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.775653 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.775714 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.775674 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-229.ec2.internal\": node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:42.796448 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.796429 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:42.850552 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.850525 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal"] Jul 10 17:38:42.850642 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.850591 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 17:38:42.854674 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.854657 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 17:38:42.854742 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.854683 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 17:38:42.854742 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.854694 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 17:38:42.857058 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857046 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 17:38:42.857195 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857167 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.857241 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857212 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 17:38:42.857768 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857750 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 17:38:42.857768 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857761 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 17:38:42.857900 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857777 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 17:38:42.857900 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857787 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 17:38:42.857900 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857792 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 17:38:42.857900 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.857800 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 17:38:42.860109 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.860093 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.860204 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.860117 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 17:38:42.860752 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.860736 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientMemory" Jul 10 17:38:42.860831 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.860755 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 17:38:42.860831 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.860770 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeHasSufficientPID" Jul 10 17:38:42.883535 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.883515 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-229.ec2.internal\" not found" node="ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.887844 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.887830 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-229.ec2.internal\" not found" node="ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.897182 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.897167 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:42.900163 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.900144 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.900215 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.900181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.900215 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:42.900208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6cda58f405aaf9348408762755bc73-config\") pod \"kube-apiserver-proxy-ip-10-0-131-229.ec2.internal\" (UID: \"fe6cda58f405aaf9348408762755bc73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 17:38:42.997309 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:42.997289 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.000566 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.000532 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.000566 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.000557 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6cda58f405aaf9348408762755bc73-config\") pod \"kube-apiserver-proxy-ip-10-0-131-229.ec2.internal\" (UID: \"fe6cda58f405aaf9348408762755bc73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.000687 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.000572 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.000687 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.000608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.000687 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.000649 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fe6cda58f405aaf9348408762755bc73-config\") pod \"kube-apiserver-proxy-ip-10-0-131-229.ec2.internal\" (UID: \"fe6cda58f405aaf9348408762755bc73\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.000687 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.000651 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa60a3a8ee1eea11634837cc945056ef-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal\" (UID: \"aa60a3a8ee1eea11634837cc945056ef\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.097948 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:43.097910 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.186312 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.186279 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.191442 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.191214 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.198022 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:43.197999 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.298514 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:43.298424 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.398868 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:43.398842 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.499381 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:43.499355 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.520661 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.520646 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 17:38:43.520788 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.520772 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 17:38:43.599997 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:43.599913 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.611993 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.611954 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 17:33:42 +0000 UTC" deadline="2027-12-21 23:11:27.601279179 +0000 UTC" Jul 10 17:38:43.612044 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.611991 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12701h32m43.989291296s" Jul 10 17:38:43.700507 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:43.700475 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.751646 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.751622 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 17:38:43.800824 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:43.800801 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-229.ec2.internal\" not found" Jul 10 17:38:43.814960 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:43.814916 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa60a3a8ee1eea11634837cc945056ef.slice/crio-d64584fccafa33f75fd0d23383192c6fb86e2c749db96d9d35fe1c7c932c3253 WatchSource:0}: Error finding container d64584fccafa33f75fd0d23383192c6fb86e2c749db96d9d35fe1c7c932c3253: Status 404 returned error can't find the container with id d64584fccafa33f75fd0d23383192c6fb86e2c749db96d9d35fe1c7c932c3253 Jul 10 17:38:43.815192 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:43.815173 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe6cda58f405aaf9348408762755bc73.slice/crio-4d56d64a612a00c042a6a860ca21cd0953221e7590829398cb4a88b7e36f35fd WatchSource:0}: Error finding container 4d56d64a612a00c042a6a860ca21cd0953221e7590829398cb4a88b7e36f35fd: Status 404 returned error can't find the container with id 4d56d64a612a00c042a6a860ca21cd0953221e7590829398cb4a88b7e36f35fd Jul 10 17:38:43.819401 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.819381 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 17:38:43.857637 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.857621 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 17:38:43.897525 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.897501 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.911785 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.911769 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 17:38:43.913185 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.913173 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" Jul 10 17:38:43.921126 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:43.921110 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 17:38:44.079259 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.079235 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 17:38:44.482791 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.482762 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 17:38:44.572771 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.572478 2568 apiserver.go:52] "Watching apiserver" Jul 10 17:38:44.585471 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.585439 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 17:38:44.585987 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.585959 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-rkvwl","openshift-image-registry/node-ca-zg66n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal","openshift-multus/network-metrics-daemon-l5nwq","openshift-network-diagnostics/network-check-target-r29fc","openshift-network-operator/iptables-alerter-b4mzb","openshift-ovn-kubernetes/ovnkube-node-7hnvb","kube-system/global-pull-secret-syncer-9q7pt","kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9","openshift-cluster-node-tuning-operator/tuned-zwr4m","openshift-dns/node-resolver-lklvj","openshift-multus/multus-additional-cni-plugins-h2zbp","openshift-multus/multus-fpjvn"] Jul 10 17:38:44.589141 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.589114 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.591745 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.591699 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.592758 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.592737 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 17:38:44.592904 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.592805 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 17:38:44.593016 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.592991 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 17:38:44.593089 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.593075 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 17:38:44.593143 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.593104 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 17:38:44.593317 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.593300 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 17:38:44.593908 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.593666 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nswpd\"" Jul 10 17:38:44.595089 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.594217 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 17:38:44.595089 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.594528 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 17:38:44.595089 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.594843 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 17:38:44.595306 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.595175 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 17:38:44.595713 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.595690 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 17:38:44.596422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.596137 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dp9rw\"" Jul 10 17:38:44.597136 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.597109 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.597990 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.597949 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 17:38:44.600517 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.599924 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:44.600517 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.600017 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:44.601369 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.601124 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 17:38:44.601369 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.601197 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nkqhq\"" Jul 10 17:38:44.602819 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.602271 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:44.602819 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.602344 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:38:44.605767 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.604990 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.608528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.607898 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 17:38:44.608528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.608179 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gdrv8\"" Jul 10 17:38:44.608528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.608311 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 17:38:44.608528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.608390 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 17:38:44.608782 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.608546 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.608782 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.608630 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:38:44.610131 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610100 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-ovnkube-config\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610243 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-cni-bin\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.610243 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610174 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-ovnkube-script-lib\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610243 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610203 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdnhw\" (UniqueName: \"kubernetes.io/projected/5fc85324-2152-4dfd-8ace-99be0e85b860-kube-api-access-zdnhw\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610243 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610240 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-system-cni-dir\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.610456 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610272 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-os-release\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.610456 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610300 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-os-release\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.610456 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610324 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-cni-binary-copy\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.610456 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-cni-netd\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610456 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610378 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.610456 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-var-lib-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610456 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610429 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-multus-certs\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610466 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-etc-kubernetes\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610491 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-systemd\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610556 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-run-ovn-kubernetes\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610583 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-cni-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610608 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-cni-multus\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610639 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-log-socket\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610666 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-env-overrides\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610722 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cni-binary-copy\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg756\" (UniqueName: \"kubernetes.io/projected/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-kube-api-access-fg756\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.610848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610794 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-system-cni-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610869 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-daemon-config\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610906 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610932 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fc85324-2152-4dfd-8ace-99be0e85b860-ovn-node-metrics-cert\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.610958 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cnibin\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-kubelet\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611069 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-hostroot\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-conf-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-slash\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-run-netns\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611193 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-cnibin\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611210 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611216 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-k8s-cni-cncf-io\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611250 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-netns\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611274 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7577\" (UniqueName: \"kubernetes.io/projected/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-kube-api-access-z7577\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-ovn\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611324 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.611459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.612321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-socket-dir-parent\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.612321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq767\" (UniqueName: \"kubernetes.io/projected/d209f2ef-b7c8-4817-8950-548fbf79929d-kube-api-access-cq767\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:44.612321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-kubelet\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.612321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-systemd-units\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.612321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611479 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-etc-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.612321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611494 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-node-log\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.612321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.611509 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-cni-bin\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.612821 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.612734 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 17:38:44.614008 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.613900 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 17:38:44.614256 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.614238 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6l57c\"" Jul 10 17:38:44.615291 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.615269 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 17:38:44.615586 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.615564 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.618179 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.618848 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.619069 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.619287 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z8qz6\"" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.619744 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.622940 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.623018 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.623299 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 17:38:44.623314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.623317 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 17:38:44.625003 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.623860 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9rnfm\"" Jul 10 17:38:44.626282 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.625357 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.626282 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.626220 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6rf7x\"" Jul 10 17:38:44.626282 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.626220 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 17:38:44.626532 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.626472 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 17:38:44.632814 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.632787 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 17:38:44.632980 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.632961 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glwbs\"" Jul 10 17:38:44.633203 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.633185 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 17:38:44.636780 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.636755 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lchfz" Jul 10 17:38:44.641241 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.641221 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lchfz" Jul 10 17:38:44.699367 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.699344 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 17:38:44.711794 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.711771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-hostroot\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.711899 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.711807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-run-netns\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.711899 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.711872 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-run-netns\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.711899 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.711873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-hostroot\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.712133 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.711901 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll7zm\" (UniqueName: \"kubernetes.io/projected/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-kube-api-access-ll7zm\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.712133 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.711933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-socket-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.712133 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.711951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:44.712133 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.711994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-netns\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.712133 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712050 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-socket-dir-parent\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.712133 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712108 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-netns\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712154 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-kubelet\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712196 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-socket-dir-parent\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712230 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-etc-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712257 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-node-log\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712301 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-kubelet\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712337 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-node-log\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712336 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-cni-bin\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712407 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-ovnkube-config\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712412 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-cni-bin\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-system-cni-dir\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712453 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-etc-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.712488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712485 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-os-release\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712624 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-system-cni-dir\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712644 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-os-release\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-cni-bin\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712702 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-iptables-alerter-script\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712729 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712753 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-modprobe-d\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712797 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-cni-bin\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysctl-conf\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712834 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-systemd\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712884 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-os-release\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712906 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712931 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-registration-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-run\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.712989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.712994 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-os-release\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713000 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-var-lib-kubelet\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713027 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-tuned\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/329c89c4-cfec-4be0-a5a8-aae54e2381b6-kubelet-config\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713065 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-var-lib-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713077 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-ovnkube-config\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-multus-certs\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713115 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-multus-certs\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713119 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-etc-kubernetes\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713153 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-var-lib-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713170 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-systemd\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713188 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-etc-kubernetes\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-run-ovn-kubernetes\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-log-socket\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713245 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cni-binary-copy\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713250 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-systemd\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.713692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713274 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-run-ovn-kubernetes\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg756\" (UniqueName: \"kubernetes.io/projected/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-kube-api-access-fg756\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713301 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-log-socket\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713329 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-tmp-dir\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713362 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-system-cni-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-daemon-config\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713418 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fc85324-2152-4dfd-8ace-99be0e85b860-ovn-node-metrics-cert\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713444 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cnibin\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713471 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bcd26b75-8c01-4a03-ba75-ba63b39fe0bb-konnectivity-ca\") pod \"konnectivity-agent-rkvwl\" (UID: \"bcd26b75-8c01-4a03-ba75-ba63b39fe0bb\") " pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-kubernetes\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713526 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-kubelet\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-conf-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-slash\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca928c87-0863-459c-bff1-2c7d3d1dc864-host\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbj2f\" (UniqueName: \"kubernetes.io/projected/ca928c87-0863-459c-bff1-2c7d3d1dc864-kube-api-access-gbj2f\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713673 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-sys-fs\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-cnibin\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.714556 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-k8s-cni-cncf-io\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713747 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7577\" (UniqueName: \"kubernetes.io/projected/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-kube-api-access-z7577\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713753 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cni-binary-copy\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713774 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-ovn\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-slash\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713808 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713843 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-kubelet\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713864 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-hosts-file\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713893 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-cnibin\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-etc-selinux\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713940 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq767\" (UniqueName: \"kubernetes.io/projected/d209f2ef-b7c8-4817-8950-548fbf79929d-kube-api-access-cq767\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713941 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-run-k8s-cni-cncf-io\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.713957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-systemd-units\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714029 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-systemd-units\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714084 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-conf-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714119 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cnibin\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.715410 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714158 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-ovn\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdnhw\" (UniqueName: \"kubernetes.io/projected/5fc85324-2152-4dfd-8ace-99be0e85b860-kube-api-access-zdnhw\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714265 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwv8b\" (UniqueName: \"kubernetes.io/projected/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-kube-api-access-bwv8b\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714331 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysconfig\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714412 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-lib-modules\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714446 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-ovnkube-script-lib\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714470 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ca928c87-0863-459c-bff1-2c7d3d1dc864-serviceca\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714495 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bcd26b75-8c01-4a03-ba75-ba63b39fe0bb-agent-certs\") pod \"konnectivity-agent-rkvwl\" (UID: \"bcd26b75-8c01-4a03-ba75-ba63b39fe0bb\") " pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714521 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-device-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714522 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714579 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvdkt\" (UniqueName: \"kubernetes.io/projected/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-kube-api-access-qvdkt\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714609 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9485c\" (UniqueName: \"kubernetes.io/projected/82e324f6-2a63-45f0-87c6-b67ae513f9c1-kube-api-access-9485c\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714613 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-system-cni-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714646 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-cni-binary-copy\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714679 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-cni-netd\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-sys\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.716418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-daemon-config\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714766 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-host\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714805 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-cni-netd\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714893 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e324f6-2a63-45f0-87c6-b67ae513f9c1-tmp\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.714932 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-run-openvswitch\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715000 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/329c89c4-cfec-4be0-a5a8-aae54e2381b6-dbus\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715043 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715081 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-cni-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715121 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-cni-multus\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715155 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-ovnkube-script-lib\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715150 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-multus-cni-dir\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715181 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715212 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-env-overrides\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715214 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-host-var-lib-cni-multus\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715239 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fc85324-2152-4dfd-8ace-99be0e85b860-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.715317 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:44.717319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715358 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-host-slash\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.718361 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715391 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-cni-binary-copy\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.718361 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.715398 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:38:45.215360625 +0000 UTC m=+3.043009382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:44.718361 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysctl-d\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.718361 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.715657 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fc85324-2152-4dfd-8ace-99be0e85b860-env-overrides\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.719288 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.719265 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fc85324-2152-4dfd-8ace-99be0e85b860-ovn-node-metrics-cert\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.731747 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.731715 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg756\" (UniqueName: \"kubernetes.io/projected/0108a2b0-af54-4d1b-804c-21f0fa2e8c61-kube-api-access-fg756\") pod \"multus-additional-cni-plugins-h2zbp\" (UID: \"0108a2b0-af54-4d1b-804c-21f0fa2e8c61\") " pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.733398 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.733342 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7577\" (UniqueName: \"kubernetes.io/projected/a763cdaa-9c2e-4c40-adef-857b7ded6ffe-kube-api-access-z7577\") pod \"multus-fpjvn\" (UID: \"a763cdaa-9c2e-4c40-adef-857b7ded6ffe\") " pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.734234 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.734208 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq767\" (UniqueName: \"kubernetes.io/projected/d209f2ef-b7c8-4817-8950-548fbf79929d-kube-api-access-cq767\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:44.735121 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.735097 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdnhw\" (UniqueName: \"kubernetes.io/projected/5fc85324-2152-4dfd-8ace-99be0e85b860-kube-api-access-zdnhw\") pod \"ovnkube-node-7hnvb\" (UID: \"5fc85324-2152-4dfd-8ace-99be0e85b860\") " pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.754756 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.754714 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" event={"ID":"fe6cda58f405aaf9348408762755bc73","Type":"ContainerStarted","Data":"4d56d64a612a00c042a6a860ca21cd0953221e7590829398cb4a88b7e36f35fd"} Jul 10 17:38:44.755880 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.755860 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" event={"ID":"aa60a3a8ee1eea11634837cc945056ef","Type":"ContainerStarted","Data":"d64584fccafa33f75fd0d23383192c6fb86e2c749db96d9d35fe1c7c932c3253"} Jul 10 17:38:44.816664 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816641 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-iptables-alerter-script\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.816797 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816671 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.816797 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816696 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-modprobe-d\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.816797 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysctl-conf\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.816797 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816744 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-systemd\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.816797 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816770 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-registration-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.816797 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816794 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-run\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816813 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-var-lib-kubelet\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-tuned\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/329c89c4-cfec-4be0-a5a8-aae54e2381b6-kubelet-config\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816862 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-tmp-dir\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bcd26b75-8c01-4a03-ba75-ba63b39fe0bb-konnectivity-ca\") pod \"konnectivity-agent-rkvwl\" (UID: \"bcd26b75-8c01-4a03-ba75-ba63b39fe0bb\") " pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-kubernetes\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816928 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca928c87-0863-459c-bff1-2c7d3d1dc864-host\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816951 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbj2f\" (UniqueName: \"kubernetes.io/projected/ca928c87-0863-459c-bff1-2c7d3d1dc864-kube-api-access-gbj2f\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.816974 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-sys-fs\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817021 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-hosts-file\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817048 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-etc-selinux\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817084 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-var-lib-kubelet\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817125 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817137 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-systemd\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817159 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-etc-selinux\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817161 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysctl-conf\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.817750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817186 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-registration-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817238 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-run\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817296 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-modprobe-d\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817317 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/329c89c4-cfec-4be0-a5a8-aae54e2381b6-kubelet-config\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817343 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-kubernetes\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817398 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-sys-fs\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817450 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-hosts-file\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817494 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ca928c87-0863-459c-bff1-2c7d3d1dc864-host\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwv8b\" (UniqueName: \"kubernetes.io/projected/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-kube-api-access-bwv8b\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817638 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysconfig\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817666 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-lib-modules\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817696 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ca928c87-0863-459c-bff1-2c7d3d1dc864-serviceca\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bcd26b75-8c01-4a03-ba75-ba63b39fe0bb-agent-certs\") pod \"konnectivity-agent-rkvwl\" (UID: \"bcd26b75-8c01-4a03-ba75-ba63b39fe0bb\") " pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-device-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817862 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-iptables-alerter-script\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvdkt\" (UniqueName: \"kubernetes.io/projected/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-kube-api-access-qvdkt\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817909 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9485c\" (UniqueName: \"kubernetes.io/projected/82e324f6-2a63-45f0-87c6-b67ae513f9c1-kube-api-access-9485c\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817936 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-sys\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.818851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817959 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-host\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817975 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-lib-modules\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.817983 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e324f6-2a63-45f0-87c6-b67ae513f9c1-tmp\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818027 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/329c89c4-cfec-4be0-a5a8-aae54e2381b6-dbus\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818120 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-host-slash\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818144 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysctl-d\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818178 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll7zm\" (UniqueName: \"kubernetes.io/projected/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-kube-api-access-ll7zm\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818191 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bcd26b75-8c01-4a03-ba75-ba63b39fe0bb-konnectivity-ca\") pod \"konnectivity-agent-rkvwl\" (UID: \"bcd26b75-8c01-4a03-ba75-ba63b39fe0bb\") " pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818201 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-socket-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818260 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-tmp-dir\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818270 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysconfig\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818277 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-host\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818356 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-device-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818419 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-socket-dir\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818487 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-sys\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818600 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-host-slash\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.819729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818698 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ca928c87-0863-459c-bff1-2c7d3d1dc864-serviceca\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.820668 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.818830 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/329c89c4-cfec-4be0-a5a8-aae54e2381b6-dbus\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:44.820668 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.819003 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-sysctl-d\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.820668 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.819115 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:44.820668 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.819173 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret podName:329c89c4-cfec-4be0-a5a8-aae54e2381b6 nodeName:}" failed. No retries permitted until 2026-07-10 17:38:45.3191546 +0000 UTC m=+3.146803356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret") pod "global-pull-secret-syncer-9q7pt" (UID: "329c89c4-cfec-4be0-a5a8-aae54e2381b6") : object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:44.821509 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.821473 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bcd26b75-8c01-4a03-ba75-ba63b39fe0bb-agent-certs\") pod \"konnectivity-agent-rkvwl\" (UID: \"bcd26b75-8c01-4a03-ba75-ba63b39fe0bb\") " pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:38:44.821625 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.821541 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82e324f6-2a63-45f0-87c6-b67ae513f9c1-etc-tuned\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.821625 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.821586 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e324f6-2a63-45f0-87c6-b67ae513f9c1-tmp\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.826104 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.826075 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbj2f\" (UniqueName: \"kubernetes.io/projected/ca928c87-0863-459c-bff1-2c7d3d1dc864-kube-api-access-gbj2f\") pod \"node-ca-zg66n\" (UID: \"ca928c87-0863-459c-bff1-2c7d3d1dc864\") " pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.827968 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.827913 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwv8b\" (UniqueName: \"kubernetes.io/projected/253f4440-c7a5-458e-bd15-6e7b7cf9fd5e-kube-api-access-bwv8b\") pod \"iptables-alerter-b4mzb\" (UID: \"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e\") " pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.830137 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.830115 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 17:38:44.830243 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.830143 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 17:38:44.830243 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.830157 2568 projected.go:196] Error preparing data for projected volume kube-api-access-6sqb9 for pod openshift-network-diagnostics/network-check-target-r29fc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:44.830360 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:44.830242 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9 podName:7119c54b-8e6a-4735-bc5e-6e64a41f6874 nodeName:}" failed. No retries permitted until 2026-07-10 17:38:45.330216472 +0000 UTC m=+3.157865235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6sqb9" (UniqueName: "kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9") pod "network-check-target-r29fc" (UID: "7119c54b-8e6a-4735-bc5e-6e64a41f6874") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:44.833907 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.832104 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9485c\" (UniqueName: \"kubernetes.io/projected/82e324f6-2a63-45f0-87c6-b67ae513f9c1-kube-api-access-9485c\") pod \"tuned-zwr4m\" (UID: \"82e324f6-2a63-45f0-87c6-b67ae513f9c1\") " pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.834424 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.834362 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvdkt\" (UniqueName: \"kubernetes.io/projected/4890ac26-42fc-4aae-acb2-91aa94d7a8a7-kube-api-access-qvdkt\") pod \"aws-ebs-csi-driver-node-pc9v9\" (UID: \"4890ac26-42fc-4aae-acb2-91aa94d7a8a7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.834886 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.834857 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll7zm\" (UniqueName: \"kubernetes.io/projected/057ec1b6-2d2e-4a4b-baac-ddd4f33b302a-kube-api-access-ll7zm\") pod \"node-resolver-lklvj\" (UID: \"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a\") " pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.921799 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.921767 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" Jul 10 17:38:44.924616 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.924589 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:38:44.934156 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:44.934126 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0108a2b0_af54_4d1b_804c_21f0fa2e8c61.slice/crio-3db6f1ab5ba8c5d06c129256efc6d862496973d29604055d84ca99fef706fb2e WatchSource:0}: Error finding container 3db6f1ab5ba8c5d06c129256efc6d862496973d29604055d84ca99fef706fb2e: Status 404 returned error can't find the container with id 3db6f1ab5ba8c5d06c129256efc6d862496973d29604055d84ca99fef706fb2e Jul 10 17:38:44.936098 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:44.936013 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc85324_2152_4dfd_8ace_99be0e85b860.slice/crio-175bfdab0d11400bffde610b99ca26aeb133a47e54391c1d5ecbc96bb7c83dc3 WatchSource:0}: Error finding container 175bfdab0d11400bffde610b99ca26aeb133a47e54391c1d5ecbc96bb7c83dc3: Status 404 returned error can't find the container with id 175bfdab0d11400bffde610b99ca26aeb133a47e54391c1d5ecbc96bb7c83dc3 Jul 10 17:38:44.939929 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.939811 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fpjvn" Jul 10 17:38:44.945102 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.944908 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b4mzb" Jul 10 17:38:44.952084 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:44.951386 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda763cdaa_9c2e_4c40_adef_857b7ded6ffe.slice/crio-c1bf83c8e65ba4e4b3164d497bbde22cb7090ee020ddf164d04a7dbd3025958d WatchSource:0}: Error finding container c1bf83c8e65ba4e4b3164d497bbde22cb7090ee020ddf164d04a7dbd3025958d: Status 404 returned error can't find the container with id c1bf83c8e65ba4e4b3164d497bbde22cb7090ee020ddf164d04a7dbd3025958d Jul 10 17:38:44.956685 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:44.956652 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253f4440_c7a5_458e_bd15_6e7b7cf9fd5e.slice/crio-0308f14949dfee78fed656d18dc85c18c0def11266dd2b36b07184b6ac578b28 WatchSource:0}: Error finding container 0308f14949dfee78fed656d18dc85c18c0def11266dd2b36b07184b6ac578b28: Status 404 returned error can't find the container with id 0308f14949dfee78fed656d18dc85c18c0def11266dd2b36b07184b6ac578b28 Jul 10 17:38:44.961507 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.961455 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:38:44.969567 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.969500 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zg66n" Jul 10 17:38:44.972542 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:44.972514 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd26b75_8c01_4a03_ba75_ba63b39fe0bb.slice/crio-124ca4e769237dc7cb20e53c16bcaa7527fe844b267a8d58db2781d405b00178 WatchSource:0}: Error finding container 124ca4e769237dc7cb20e53c16bcaa7527fe844b267a8d58db2781d405b00178: Status 404 returned error can't find the container with id 124ca4e769237dc7cb20e53c16bcaa7527fe844b267a8d58db2781d405b00178 Jul 10 17:38:44.979015 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.978965 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" Jul 10 17:38:44.980012 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:44.979987 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca928c87_0863_459c_bff1_2c7d3d1dc864.slice/crio-4d7cc6f0b8311518030880ea2c8f285f60feb89f74265324ac039c037e322d8c WatchSource:0}: Error finding container 4d7cc6f0b8311518030880ea2c8f285f60feb89f74265324ac039c037e322d8c: Status 404 returned error can't find the container with id 4d7cc6f0b8311518030880ea2c8f285f60feb89f74265324ac039c037e322d8c Jul 10 17:38:44.987524 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.987502 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" Jul 10 17:38:44.989913 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:44.989878 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4890ac26_42fc_4aae_acb2_91aa94d7a8a7.slice/crio-1a509f3a2a66475c4300c03af3ce5364bbdcc733efb59735f2854d082774c1a8 WatchSource:0}: Error finding container 1a509f3a2a66475c4300c03af3ce5364bbdcc733efb59735f2854d082774c1a8: Status 404 returned error can't find the container with id 1a509f3a2a66475c4300c03af3ce5364bbdcc733efb59735f2854d082774c1a8 Jul 10 17:38:44.995178 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:44.995156 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lklvj" Jul 10 17:38:44.997776 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:44.997748 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e324f6_2a63_45f0_87c6_b67ae513f9c1.slice/crio-d0dfa70991f106f7d12de248e4a1b7146f86e113a00b54a57b7fafcbe6cefb9e WatchSource:0}: Error finding container d0dfa70991f106f7d12de248e4a1b7146f86e113a00b54a57b7fafcbe6cefb9e: Status 404 returned error can't find the container with id d0dfa70991f106f7d12de248e4a1b7146f86e113a00b54a57b7fafcbe6cefb9e Jul 10 17:38:45.005861 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:38:45.005839 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057ec1b6_2d2e_4a4b_baac_ddd4f33b302a.slice/crio-ad6d80aa179b37138e74d2afcef47c94fd499acd0706469cfbdb23955408fefa WatchSource:0}: Error finding container ad6d80aa179b37138e74d2afcef47c94fd499acd0706469cfbdb23955408fefa: Status 404 returned error can't find the container with id ad6d80aa179b37138e74d2afcef47c94fd499acd0706469cfbdb23955408fefa Jul 10 17:38:45.222119 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.221979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:45.222329 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.222172 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:45.222329 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.222261 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:38:46.222236536 +0000 UTC m=+4.049885290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:45.323056 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.322954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:45.323230 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.323147 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:45.323230 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.323209 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret podName:329c89c4-cfec-4be0-a5a8-aae54e2381b6 nodeName:}" failed. No retries permitted until 2026-07-10 17:38:46.32319376 +0000 UTC m=+4.150842512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret") pod "global-pull-secret-syncer-9q7pt" (UID: "329c89c4-cfec-4be0-a5a8-aae54e2381b6") : object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:45.424096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.424038 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:45.424271 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.424224 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 17:38:45.424271 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.424251 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 17:38:45.424271 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.424264 2568 projected.go:196] Error preparing data for projected volume kube-api-access-6sqb9 for pod openshift-network-diagnostics/network-check-target-r29fc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:45.424445 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.424321 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9 podName:7119c54b-8e6a-4735-bc5e-6e64a41f6874 nodeName:}" failed. No retries permitted until 2026-07-10 17:38:46.424306731 +0000 UTC m=+4.251955479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sqb9" (UniqueName: "kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9") pod "network-check-target-r29fc" (UID: "7119c54b-8e6a-4735-bc5e-6e64a41f6874") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:45.644539 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.644501 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 17:33:44 +0000 UTC" deadline="2028-01-05 09:39:58.486711121 +0000 UTC" Jul 10 17:38:45.644539 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.644534 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13048h1m12.842180704s" Jul 10 17:38:45.750778 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.750581 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:45.750937 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.750778 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:45.750937 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.750846 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:38:45.751270 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:45.751245 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:45.758943 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.758885 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b4mzb" event={"ID":"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e","Type":"ContainerStarted","Data":"0308f14949dfee78fed656d18dc85c18c0def11266dd2b36b07184b6ac578b28"} Jul 10 17:38:45.760306 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.760281 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"175bfdab0d11400bffde610b99ca26aeb133a47e54391c1d5ecbc96bb7c83dc3"} Jul 10 17:38:45.761737 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.761683 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rkvwl" event={"ID":"bcd26b75-8c01-4a03-ba75-ba63b39fe0bb","Type":"ContainerStarted","Data":"124ca4e769237dc7cb20e53c16bcaa7527fe844b267a8d58db2781d405b00178"} Jul 10 17:38:45.763028 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.763001 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fpjvn" event={"ID":"a763cdaa-9c2e-4c40-adef-857b7ded6ffe","Type":"ContainerStarted","Data":"c1bf83c8e65ba4e4b3164d497bbde22cb7090ee020ddf164d04a7dbd3025958d"} Jul 10 17:38:45.764222 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.764197 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" event={"ID":"0108a2b0-af54-4d1b-804c-21f0fa2e8c61","Type":"ContainerStarted","Data":"3db6f1ab5ba8c5d06c129256efc6d862496973d29604055d84ca99fef706fb2e"} Jul 10 17:38:45.765427 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.765404 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lklvj" event={"ID":"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a","Type":"ContainerStarted","Data":"ad6d80aa179b37138e74d2afcef47c94fd499acd0706469cfbdb23955408fefa"} Jul 10 17:38:45.766615 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.766588 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" event={"ID":"82e324f6-2a63-45f0-87c6-b67ae513f9c1","Type":"ContainerStarted","Data":"d0dfa70991f106f7d12de248e4a1b7146f86e113a00b54a57b7fafcbe6cefb9e"} Jul 10 17:38:45.767782 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.767761 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" event={"ID":"4890ac26-42fc-4aae-acb2-91aa94d7a8a7","Type":"ContainerStarted","Data":"1a509f3a2a66475c4300c03af3ce5364bbdcc733efb59735f2854d082774c1a8"} Jul 10 17:38:45.768827 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:45.768806 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zg66n" event={"ID":"ca928c87-0863-459c-bff1-2c7d3d1dc864","Type":"ContainerStarted","Data":"4d7cc6f0b8311518030880ea2c8f285f60feb89f74265324ac039c037e322d8c"} Jul 10 17:38:46.230761 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:46.230719 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:46.230957 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.230936 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:46.231027 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.230998 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:38:48.230982468 +0000 UTC m=+6.058631217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:46.332359 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:46.332217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:46.332359 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.332330 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:46.332497 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.332387 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret podName:329c89c4-cfec-4be0-a5a8-aae54e2381b6 nodeName:}" failed. No retries permitted until 2026-07-10 17:38:48.332373501 +0000 UTC m=+6.160022249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret") pod "global-pull-secret-syncer-9q7pt" (UID: "329c89c4-cfec-4be0-a5a8-aae54e2381b6") : object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:46.433184 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:46.432589 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:46.433184 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.432741 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 17:38:46.433184 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.432761 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 17:38:46.433184 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.432776 2568 projected.go:196] Error preparing data for projected volume kube-api-access-6sqb9 for pod openshift-network-diagnostics/network-check-target-r29fc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:46.433184 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.432832 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9 podName:7119c54b-8e6a-4735-bc5e-6e64a41f6874 nodeName:}" failed. No retries permitted until 2026-07-10 17:38:48.432811213 +0000 UTC m=+6.260459960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sqb9" (UniqueName: "kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9") pod "network-check-target-r29fc" (UID: "7119c54b-8e6a-4735-bc5e-6e64a41f6874") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:46.645806 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:46.645735 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 17:33:44 +0000 UTC" deadline="2028-04-26 06:18:45.06295769 +0000 UTC" Jul 10 17:38:46.645806 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:46.645802 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15732h39m58.417160364s" Jul 10 17:38:46.752983 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:46.752864 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:46.753174 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:46.753006 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:38:46.784525 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:46.784485 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" event={"ID":"fe6cda58f405aaf9348408762755bc73","Type":"ContainerStarted","Data":"c854995bccc5a0b0362bf61d7cb3ada370c876cdb5286b620b2b4d1ce61c7ad9"} Jul 10 17:38:47.750596 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:47.750565 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:47.751124 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:47.750701 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:47.751124 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:47.751080 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:47.751230 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:47.751175 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:38:47.791768 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:47.791725 2568 generic.go:356] "Generic (PLEG): container finished" podID="aa60a3a8ee1eea11634837cc945056ef" containerID="59a4a9a8e40f001b2dfaa69409694cf35a934c71787454c637aedb557af88ba8" exitCode=0 Jul 10 17:38:47.792467 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:47.792199 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" event={"ID":"aa60a3a8ee1eea11634837cc945056ef","Type":"ContainerDied","Data":"59a4a9a8e40f001b2dfaa69409694cf35a934c71787454c637aedb557af88ba8"} Jul 10 17:38:47.807997 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:47.807931 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-229.ec2.internal" podStartSLOduration=3.807910792 podStartE2EDuration="3.807910792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:38:46.801989301 +0000 UTC m=+4.629638075" watchObservedRunningTime="2026-07-10 17:38:47.807910792 +0000 UTC m=+5.635559561" Jul 10 17:38:48.254399 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:48.254355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:48.254578 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.254520 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:48.254638 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.254591 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:38:52.254570228 +0000 UTC m=+10.082218985 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:48.355267 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:48.355237 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:48.355427 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.355419 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:48.355483 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.355475 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret podName:329c89c4-cfec-4be0-a5a8-aae54e2381b6 nodeName:}" failed. No retries permitted until 2026-07-10 17:38:52.355457553 +0000 UTC m=+10.183106320 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret") pod "global-pull-secret-syncer-9q7pt" (UID: "329c89c4-cfec-4be0-a5a8-aae54e2381b6") : object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:48.455847 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:48.455811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:48.456035 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.456004 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 17:38:48.456035 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.456024 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 17:38:48.456035 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.456037 2568 projected.go:196] Error preparing data for projected volume kube-api-access-6sqb9 for pod openshift-network-diagnostics/network-check-target-r29fc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:48.456208 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.456091 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9 podName:7119c54b-8e6a-4735-bc5e-6e64a41f6874 nodeName:}" failed. No retries permitted until 2026-07-10 17:38:52.456073424 +0000 UTC m=+10.283722177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sqb9" (UniqueName: "kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9") pod "network-check-target-r29fc" (UID: "7119c54b-8e6a-4735-bc5e-6e64a41f6874") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:48.752748 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:48.752193 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:48.752748 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:48.752319 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:38:48.803271 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:48.802586 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" event={"ID":"aa60a3a8ee1eea11634837cc945056ef","Type":"ContainerStarted","Data":"78e330a0971f243245ca5f1fe04eb21e706b97ffe5d0154dbfa6ece06b4fcd3b"} Jul 10 17:38:48.825986 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:48.823056 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-229.ec2.internal" podStartSLOduration=4.823035839 podStartE2EDuration="4.823035839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:38:48.82214436 +0000 UTC m=+6.649793131" watchObservedRunningTime="2026-07-10 17:38:48.823035839 +0000 UTC m=+6.650684607" Jul 10 17:38:49.750610 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:49.750089 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:49.750610 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:49.750094 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:49.750610 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:49.750239 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:49.750610 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:49.750308 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:38:50.751042 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:50.750538 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:50.751042 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:50.750671 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:38:51.750796 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:51.750761 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:51.750962 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:51.750763 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:51.750962 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:51.750876 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:51.751096 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:51.750986 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:38:52.288725 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:52.288683 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:52.288911 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.288866 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:52.288989 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.288947 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:39:00.28892682 +0000 UTC m=+18.116575579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:38:52.389441 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:52.389403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:52.389621 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.389523 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:52.389621 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.389577 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret podName:329c89c4-cfec-4be0-a5a8-aae54e2381b6 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:00.38956319 +0000 UTC m=+18.217211936 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret") pod "global-pull-secret-syncer-9q7pt" (UID: "329c89c4-cfec-4be0-a5a8-aae54e2381b6") : object "kube-system"/"original-pull-secret" not registered Jul 10 17:38:52.490027 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:52.489990 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:52.490209 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.490187 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 17:38:52.490254 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.490220 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 17:38:52.490254 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.490233 2568 projected.go:196] Error preparing data for projected volume kube-api-access-6sqb9 for pod openshift-network-diagnostics/network-check-target-r29fc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:52.490314 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.490294 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9 podName:7119c54b-8e6a-4735-bc5e-6e64a41f6874 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:00.4902753 +0000 UTC m=+18.317924046 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sqb9" (UniqueName: "kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9") pod "network-check-target-r29fc" (UID: "7119c54b-8e6a-4735-bc5e-6e64a41f6874") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:38:52.751874 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:52.751842 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:52.752305 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:52.752218 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:38:53.749932 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:53.749901 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:53.750119 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:53.750041 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:38:53.750119 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:53.750051 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:53.750241 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:53.750147 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:54.753040 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:54.753005 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:54.753497 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:54.753128 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:38:55.750592 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:55.750560 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:55.750760 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:55.750688 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:38:55.751017 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:55.750560 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:55.751147 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:55.751123 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:56.750531 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:56.750497 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:56.750947 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:56.750633 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:38:57.750074 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:57.750042 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:57.750074 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:57.750070 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:57.750286 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:57.750171 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:38:57.750347 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:57.750293 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:58.750335 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:58.750307 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:38:58.750780 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:58.750418 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:38:59.750794 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:59.750754 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:38:59.751340 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:38:59.750753 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:38:59.751340 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:59.750901 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:38:59.751340 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:38:59.750983 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:00.349299 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:00.349252 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:00.349527 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.349428 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:39:00.349527 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.349518 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:39:16.349493695 +0000 UTC m=+34.177142472 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:39:00.450296 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:00.450260 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:00.450486 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.450443 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 17:39:00.450548 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.450509 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret podName:329c89c4-cfec-4be0-a5a8-aae54e2381b6 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:16.450493889 +0000 UTC m=+34.278142637 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret") pod "global-pull-secret-syncer-9q7pt" (UID: "329c89c4-cfec-4be0-a5a8-aae54e2381b6") : object "kube-system"/"original-pull-secret" not registered Jul 10 17:39:00.550983 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:00.550929 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:00.551162 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.551114 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 17:39:00.551162 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.551137 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 17:39:00.551162 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.551149 2568 projected.go:196] Error preparing data for projected volume kube-api-access-6sqb9 for pod openshift-network-diagnostics/network-check-target-r29fc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:39:00.551318 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.551218 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9 podName:7119c54b-8e6a-4735-bc5e-6e64a41f6874 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:16.551197211 +0000 UTC m=+34.378845981 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sqb9" (UniqueName: "kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9") pod "network-check-target-r29fc" (UID: "7119c54b-8e6a-4735-bc5e-6e64a41f6874") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:39:00.750660 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:00.750627 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:00.750821 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:00.750770 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:01.750798 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:01.750759 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:01.751055 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:01.750770 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:01.751055 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:01.750902 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:01.751055 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:01.751009 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:02.751446 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:02.751407 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:02.751899 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:02.751521 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:03.750658 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:03.750622 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:03.750658 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:03.750648 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:03.750868 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:03.750739 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:03.750950 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:03.750906 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:04.750474 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:04.750428 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:04.750923 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:04.750563 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:05.750989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.750685 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:05.751567 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.750685 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:05.751567 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:05.751096 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:05.751567 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:05.751187 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:05.838217 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.838142 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rkvwl" event={"ID":"bcd26b75-8c01-4a03-ba75-ba63b39fe0bb","Type":"ContainerStarted","Data":"e9c90795c0cee34efdb1eb0b7f6fc4b946a5c0393a0daa0b70cf3e025053513b"} Jul 10 17:39:05.840100 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.840068 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fpjvn" event={"ID":"a763cdaa-9c2e-4c40-adef-857b7ded6ffe","Type":"ContainerStarted","Data":"814b8c4a470d93b1681a47913caea2fe6eb29b5f210cb2e5f43c46a196f42f0c"} Jul 10 17:39:05.841789 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.841764 2568 generic.go:356] "Generic (PLEG): container finished" podID="0108a2b0-af54-4d1b-804c-21f0fa2e8c61" containerID="331a3dcbd4c2d0213b080c0918c303633ca6d61063007b4e9c32d21c8c59b787" exitCode=0 Jul 10 17:39:05.841916 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.841844 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" event={"ID":"0108a2b0-af54-4d1b-804c-21f0fa2e8c61","Type":"ContainerDied","Data":"331a3dcbd4c2d0213b080c0918c303633ca6d61063007b4e9c32d21c8c59b787"} Jul 10 17:39:05.843643 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.843618 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lklvj" event={"ID":"057ec1b6-2d2e-4a4b-baac-ddd4f33b302a","Type":"ContainerStarted","Data":"c0f90753437de450ed5d85c8c6600559d6dcc2e6b7e760380fc0fce6f2414aeb"} Jul 10 17:39:05.845334 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.845310 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" event={"ID":"82e324f6-2a63-45f0-87c6-b67ae513f9c1","Type":"ContainerStarted","Data":"c94ac25ef0b4339e5ab2cf44d4cb42a4b94089eed5eec51fa9a0feb12f5f35f2"} Jul 10 17:39:05.846988 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.846948 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" event={"ID":"4890ac26-42fc-4aae-acb2-91aa94d7a8a7","Type":"ContainerStarted","Data":"87a62a53d1541bd3766fa744ea317715e2dd6294084ace043343926ea72cd6e6"} Jul 10 17:39:05.848608 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.848580 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zg66n" event={"ID":"ca928c87-0863-459c-bff1-2c7d3d1dc864","Type":"ContainerStarted","Data":"0bbb926782d69f745a47a4c698a302b760d0daca4e051eea5b65204c09e9a7a0"} Jul 10 17:39:05.851683 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.851665 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:39:05.852056 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.852008 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rkvwl" podStartSLOduration=2.956264924 podStartE2EDuration="22.851994026s" totalImagesPullingTime="19.895729102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:05.851438102 +0000 UTC m=+23.679086869" watchObservedRunningTime="2026-07-10 17:39:05.851994026 +0000 UTC m=+23.679642795" Jul 10 17:39:05.852174 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.852110 2568 generic.go:356] "Generic (PLEG): container finished" podID="5fc85324-2152-4dfd-8ace-99be0e85b860" containerID="95a280021188d8630a435e2c6eaa093b29c0088eff2fdfb26b823290c364f376" exitCode=1 Jul 10 17:39:05.852174 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.852139 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"903c41bd6221f5767363d132efa0a3901be2922b14317a8ae83d4d4781c213b6"} Jul 10 17:39:05.852278 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.852206 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"5bc01866a7271aad7455e5e9b1ecc130d87b5e9f627522c7bbac169f38dae333"} Jul 10 17:39:05.852278 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.852255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"dbdfd52150edaa1232ad374d01de0ab68121efca7d227c0d77964064e5596616"} Jul 10 17:39:05.852370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.852301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"a5457a67d4edc385d8abb4525a0efc026d19a1af02d2de4a78dc4a19402398e9"} Jul 10 17:39:05.852370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.852344 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerDied","Data":"95a280021188d8630a435e2c6eaa093b29c0088eff2fdfb26b823290c364f376"} Jul 10 17:39:05.852479 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.852374 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"acab3ba49e0bb93f156310d326dd2532de0cff4db2547bdaa6d5fc93b5dc6609"} Jul 10 17:39:05.866603 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.866556 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fpjvn" podStartSLOduration=2.8327377609999997 podStartE2EDuration="22.866537974s" totalImagesPullingTime="20.033800213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:05.866128301 +0000 UTC m=+23.693777068" watchObservedRunningTime="2026-07-10 17:39:05.866537974 +0000 UTC m=+23.694186743" Jul 10 17:39:05.878792 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.878756 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zg66n" podStartSLOduration=2.940834772 podStartE2EDuration="22.8787464s" totalImagesPullingTime="19.937911628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:05.878561986 +0000 UTC m=+23.706210755" watchObservedRunningTime="2026-07-10 17:39:05.8787464 +0000 UTC m=+23.706395167" Jul 10 17:39:05.891840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.891794 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lklvj" podStartSLOduration=7.3178515 podStartE2EDuration="22.891784931s" totalImagesPullingTime="15.573933431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:05.891381861 +0000 UTC m=+23.719030629" watchObservedRunningTime="2026-07-10 17:39:05.891784931 +0000 UTC m=+23.719433699" Jul 10 17:39:05.933738 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:05.933698 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zwr4m" podStartSLOduration=3.013459439 podStartE2EDuration="22.933686117s" totalImagesPullingTime="19.920226678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:05.933199691 +0000 UTC m=+23.760848460" watchObservedRunningTime="2026-07-10 17:39:05.933686117 +0000 UTC m=+23.761334884" Jul 10 17:39:06.013529 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:06.013505 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:39:06.014107 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:06.014081 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:39:06.735784 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:06.735668 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 17:39:06.751000 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:06.750960 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:06.751133 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:06.751109 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:06.859274 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:06.859235 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" event={"ID":"4890ac26-42fc-4aae-acb2-91aa94d7a8a7","Type":"ContainerStarted","Data":"2dd38a0603f7b24cfe80303bd1de13acf4358607c3fcb85df1a70cf495effbde"} Jul 10 17:39:06.860918 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:06.860874 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b4mzb" event={"ID":"253f4440-c7a5-458e-bd15-6e7b7cf9fd5e","Type":"ContainerStarted","Data":"fe09931871cd26b0e867bd821e90d7cb633cc6abb58f03851908aaf47cc90c1c"} Jul 10 17:39:06.874050 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:06.874008 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b4mzb" podStartSLOduration=3.961438328 podStartE2EDuration="23.87399422s" totalImagesPullingTime="19.912555892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:06.873708252 +0000 UTC m=+24.701357021" watchObservedRunningTime="2026-07-10 17:39:06.87399422 +0000 UTC m=+24.701642998" Jul 10 17:39:07.678476 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:07.678375 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T17:39:06.735686227Z","UUID":"44471716-9155-4c0c-b3bc-bc527f5f95c8","Handler":null,"Name":"","Endpoint":""} Jul 10 17:39:07.680023 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:07.680001 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 17:39:07.680154 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:07.680041 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 17:39:07.750926 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:07.750901 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:07.751057 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:07.750901 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:07.751057 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:07.751044 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:07.751185 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:07.751159 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:07.866332 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:07.866191 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:39:07.866668 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:07.866642 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"eae18b2d16f7afeb915a35758d6803f47568bd8634751f67cd3e896dd6030ab1"} Jul 10 17:39:07.866762 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:07.866686 2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 17:39:08.750415 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:08.750386 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:08.750586 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:08.750512 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:08.870874 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:08.870842 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" event={"ID":"4890ac26-42fc-4aae-acb2-91aa94d7a8a7","Type":"ContainerStarted","Data":"ef52719474bac6f9ecc80637a6ad2e04b1c9835e1c6075062c33a9fb639ca141"} Jul 10 17:39:08.887196 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:08.887140 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pc9v9" podStartSLOduration=3.112988819 podStartE2EDuration="25.887125385s" totalImagesPullingTime="22.774136566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:08.887060738 +0000 UTC m=+26.714709509" watchObservedRunningTime="2026-07-10 17:39:08.887125385 +0000 UTC m=+26.714774153" Jul 10 17:39:09.750401 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:09.750366 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:09.750566 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:09.750411 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:09.750566 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:09.750488 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:09.750654 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:09.750612 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:09.893260 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:09.893227 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:39:09.893703 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:09.893367 2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 17:39:09.894017 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:09.893992 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rkvwl" Jul 10 17:39:10.750783 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:10.750749 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:10.750936 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:10.750879 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:11.750191 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.750030 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:11.750653 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.750029 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:11.750653 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:11.750263 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:11.750653 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:11.750346 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:11.880533 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.880510 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:39:11.880903 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.880880 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"401fe9051df9e8d47bf0701fab1fcc8fa636f042fa17b28418f841bb34123e7f"} Jul 10 17:39:11.881163 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.881136 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:39:11.881369 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.881352 2568 scope.go:117] "RemoveContainer" containerID="95a280021188d8630a435e2c6eaa093b29c0088eff2fdfb26b823290c364f376" Jul 10 17:39:11.883024 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.882997 2568 generic.go:356] "Generic (PLEG): container finished" podID="0108a2b0-af54-4d1b-804c-21f0fa2e8c61" containerID="4481800bcf34b7a583de1dc2f9fe9e0b6428b22a4a72f591b5cc601ecb57a510" exitCode=0 Jul 10 17:39:11.883113 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.883036 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" event={"ID":"0108a2b0-af54-4d1b-804c-21f0fa2e8c61","Type":"ContainerDied","Data":"4481800bcf34b7a583de1dc2f9fe9e0b6428b22a4a72f591b5cc601ecb57a510"} Jul 10 17:39:11.898830 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:11.898807 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:39:12.751361 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:12.751332 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:12.752036 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:12.751420 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:12.891617 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:12.891592 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:39:12.892096 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:12.892029 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" event={"ID":"5fc85324-2152-4dfd-8ace-99be0e85b860","Type":"ContainerStarted","Data":"99a28d3dd194ce95578857709595abe8d6127f013494550c6fd3f986840727a8"} Jul 10 17:39:12.892524 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:12.892502 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:39:12.892636 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:12.892594 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:39:12.911553 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:12.911526 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:39:12.949260 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:12.949219 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" podStartSLOduration=9.890410426 podStartE2EDuration="29.949204899s" totalImagesPullingTime="20.058794473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:12.920264416 +0000 UTC m=+30.747913188" watchObservedRunningTime="2026-07-10 17:39:12.949204899 +0000 UTC m=+30.776853667" Jul 10 17:39:13.282704 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:13.282535 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9q7pt"] Jul 10 17:39:13.282882 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:13.282802 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:13.282950 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:13.282877 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:13.285739 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:13.285716 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r29fc"] Jul 10 17:39:13.285883 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:13.285798 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:13.285883 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:13.285870 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:13.286334 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:13.286313 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l5nwq"] Jul 10 17:39:13.286433 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:13.286403 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:13.286495 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:13.286479 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:13.896627 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:13.896589 2568 generic.go:356] "Generic (PLEG): container finished" podID="0108a2b0-af54-4d1b-804c-21f0fa2e8c61" containerID="10ac4b4eaba6a0ea87dad4dbf0307b067c128d7caeee01767c61932d17320906" exitCode=0 Jul 10 17:39:13.897114 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:13.896664 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" event={"ID":"0108a2b0-af54-4d1b-804c-21f0fa2e8c61","Type":"ContainerDied","Data":"10ac4b4eaba6a0ea87dad4dbf0307b067c128d7caeee01767c61932d17320906"} Jul 10 17:39:14.750884 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:14.750854 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:14.751068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:14.750864 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:14.751068 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:14.751013 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:14.751068 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:14.750864 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:14.751068 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:14.751053 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:14.751257 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:14.751162 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:15.902733 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:15.902693 2568 generic.go:356] "Generic (PLEG): container finished" podID="0108a2b0-af54-4d1b-804c-21f0fa2e8c61" containerID="2c675acb950af46d3ce4ce3a89e67e9af2d7e51bd7ee0930434bb73c1cbcd897" exitCode=0 Jul 10 17:39:15.903329 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:15.902773 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" event={"ID":"0108a2b0-af54-4d1b-804c-21f0fa2e8c61","Type":"ContainerDied","Data":"2c675acb950af46d3ce4ce3a89e67e9af2d7e51bd7ee0930434bb73c1cbcd897"} Jul 10 17:39:16.368658 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:16.368627 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:16.368801 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.368747 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:39:16.368801 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.368799 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:39:48.368783992 +0000 UTC m=+66.196432738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 17:39:16.468898 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:16.468870 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:16.469039 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.469023 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 17:39:16.469088 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.469078 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret podName:329c89c4-cfec-4be0-a5a8-aae54e2381b6 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:48.469063145 +0000 UTC m=+66.296711891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret") pod "global-pull-secret-syncer-9q7pt" (UID: "329c89c4-cfec-4be0-a5a8-aae54e2381b6") : object "kube-system"/"original-pull-secret" not registered Jul 10 17:39:16.569770 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:16.569742 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:16.569898 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.569849 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 17:39:16.569898 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.569866 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 17:39:16.569898 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.569878 2568 projected.go:196] Error preparing data for projected volume kube-api-access-6sqb9 for pod openshift-network-diagnostics/network-check-target-r29fc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:39:16.570052 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.569936 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9 podName:7119c54b-8e6a-4735-bc5e-6e64a41f6874 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:48.569918337 +0000 UTC m=+66.397567087 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6sqb9" (UniqueName: "kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9") pod "network-check-target-r29fc" (UID: "7119c54b-8e6a-4735-bc5e-6e64a41f6874") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 17:39:16.751005 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:16.750897 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:16.751005 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:16.750902 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:16.751221 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:16.751069 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:16.751221 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.751080 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:39:16.751221 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.751171 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r29fc" podUID="7119c54b-8e6a-4735-bc5e-6e64a41f6874" Jul 10 17:39:16.751380 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:16.751254 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9q7pt" podUID="329c89c4-cfec-4be0-a5a8-aae54e2381b6" Jul 10 17:39:17.939337 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.939313 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-229.ec2.internal" event="NodeReady" Jul 10 17:39:17.939721 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.939421 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 17:39:17.972245 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.972207 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-gl62k"] Jul 10 17:39:17.977785 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.977747 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:17.978848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.978828 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-88d54bbc5-qgdnc"] Jul 10 17:39:17.981468 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.981446 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z8xbl\"" Jul 10 17:39:17.981569 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.981453 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 10 17:39:17.981569 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.981464 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 10 17:39:17.983713 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.983362 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:17.985733 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.985710 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 17:39:17.986816 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.986057 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-gl62k"] Jul 10 17:39:17.986816 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.986149 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 17:39:17.986816 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.986373 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 17:39:17.986816 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.986719 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tc8df\"" Jul 10 17:39:17.993424 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.993397 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dlx96"] Jul 10 17:39:17.993953 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.993931 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 17:39:17.998135 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.998116 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7zqts"] Jul 10 17:39:17.998266 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:17.998248 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dlx96" Jul 10 17:39:18.000578 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.000560 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 17:39:18.001519 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.001498 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h8cns\"" Jul 10 17:39:18.001611 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.001517 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 17:39:18.001675 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.001606 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 17:39:18.002279 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.002260 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-88d54bbc5-qgdnc"] Jul 10 17:39:18.002370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.002354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.004531 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.004514 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dbkj9\"" Jul 10 17:39:18.004709 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.004691 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 17:39:18.004784 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.004768 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 17:39:18.009129 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.008913 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dlx96"] Jul 10 17:39:18.011199 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.011144 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7zqts"] Jul 10 17:39:18.082222 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082189 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b67c448-4305-4d2b-a46b-b87d9b3860b1-config-volume\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.082370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082230 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-bound-sa-token\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.082370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c85d73b0-773f-4823-a71b-c8346bc8e11e-cert\") pod \"ingress-canary-dlx96\" (UID: \"c85d73b0-773f-4823-a71b-c8346bc8e11e\") " pod="openshift-ingress-canary/ingress-canary-dlx96" Jul 10 17:39:18.082370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwnsh\" (UniqueName: \"kubernetes.io/projected/4b67c448-4305-4d2b-a46b-b87d9b3860b1-kube-api-access-wwnsh\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.082370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082308 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.082370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082336 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-trusted-ca\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.082370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082352 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9vzz\" (UniqueName: \"kubernetes.io/projected/c85d73b0-773f-4823-a71b-c8346bc8e11e-kube-api-access-j9vzz\") pod \"ingress-canary-dlx96\" (UID: \"c85d73b0-773f-4823-a71b-c8346bc8e11e\") " pod="openshift-ingress-canary/ingress-canary-dlx96" Jul 10 17:39:18.082370 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-nginx-conf\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:18.082690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082387 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-certificates\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.082690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-installation-pull-secrets\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.082690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.082690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082467 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:18.082690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082491 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/848cca2f-0571-43b3-bef7-0c67b5b013f9-ca-trust-extracted\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.082690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cpph\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-kube-api-access-5cpph\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.082690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082579 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b67c448-4305-4d2b-a46b-b87d9b3860b1-tmp-dir\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.082690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.082603 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-image-registry-private-configuration\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.183576 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183543 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b67c448-4305-4d2b-a46b-b87d9b3860b1-tmp-dir\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.183734 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-image-registry-private-configuration\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.183734 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b67c448-4305-4d2b-a46b-b87d9b3860b1-config-volume\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.183734 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183651 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-bound-sa-token\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.183734 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c85d73b0-773f-4823-a71b-c8346bc8e11e-cert\") pod \"ingress-canary-dlx96\" (UID: \"c85d73b0-773f-4823-a71b-c8346bc8e11e\") " pod="openshift-ingress-canary/ingress-canary-dlx96" Jul 10 17:39:18.183734 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183701 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwnsh\" (UniqueName: \"kubernetes.io/projected/4b67c448-4305-4d2b-a46b-b87d9b3860b1-kube-api-access-wwnsh\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.183734 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.183989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183758 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-trusted-ca\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.183989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9vzz\" (UniqueName: \"kubernetes.io/projected/c85d73b0-773f-4823-a71b-c8346bc8e11e-kube-api-access-j9vzz\") pod \"ingress-canary-dlx96\" (UID: \"c85d73b0-773f-4823-a71b-c8346bc8e11e\") " pod="openshift-ingress-canary/ingress-canary-dlx96" Jul 10 17:39:18.183989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183808 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-nginx-conf\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:18.183989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-certificates\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.184143 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183996 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b67c448-4305-4d2b-a46b-b87d9b3860b1-tmp-dir\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.184143 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.183860 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-installation-pull-secrets\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.184143 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.184071 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.184143 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.184099 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:18.184143 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.184125 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/848cca2f-0571-43b3-bef7-0c67b5b013f9-ca-trust-extracted\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.184327 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.184156 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cpph\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-kube-api-access-5cpph\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.184379 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.184363 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:39:18.184460 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.184442 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:18.684420238 +0000 UTC m=+36.512068997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:39:18.184596 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.184501 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:39:18.184596 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.184515 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:39:18.184596 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.184527 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:39:18.184596 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.184572 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:18.684555863 +0000 UTC m=+36.512204649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:39:18.184596 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.184589 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:39:18.684580283 +0000 UTC m=+36.512229035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:39:18.185266 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.185106 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/848cca2f-0571-43b3-bef7-0c67b5b013f9-ca-trust-extracted\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.185266 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.185191 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b67c448-4305-4d2b-a46b-b87d9b3860b1-config-volume\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.185817 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.185797 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-certificates\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.185908 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.185879 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-nginx-conf\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:18.186401 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.186376 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-trusted-ca\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.188733 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.188713 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c85d73b0-773f-4823-a71b-c8346bc8e11e-cert\") pod \"ingress-canary-dlx96\" (UID: \"c85d73b0-773f-4823-a71b-c8346bc8e11e\") " pod="openshift-ingress-canary/ingress-canary-dlx96" Jul 10 17:39:18.188848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.188830 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-installation-pull-secrets\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.189392 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.189343 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-image-registry-private-configuration\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.193804 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.193780 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cpph\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-kube-api-access-5cpph\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.195231 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.195208 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-bound-sa-token\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.195340 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.195283 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwnsh\" (UniqueName: \"kubernetes.io/projected/4b67c448-4305-4d2b-a46b-b87d9b3860b1-kube-api-access-wwnsh\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.195566 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.195548 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9vzz\" (UniqueName: \"kubernetes.io/projected/c85d73b0-773f-4823-a71b-c8346bc8e11e-kube-api-access-j9vzz\") pod \"ingress-canary-dlx96\" (UID: \"c85d73b0-773f-4823-a71b-c8346bc8e11e\") " pod="openshift-ingress-canary/ingress-canary-dlx96" Jul 10 17:39:18.315291 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.315261 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dlx96" Jul 10 17:39:18.469843 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.469617 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dlx96"] Jul 10 17:39:18.473120 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:39:18.473096 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85d73b0_773f_4823_a71b_c8346bc8e11e.slice/crio-c504ebe49ae6d2634d36ed2f6f3dcef4edb8cb82293411dad269b54228093a92 WatchSource:0}: Error finding container c504ebe49ae6d2634d36ed2f6f3dcef4edb8cb82293411dad269b54228093a92: Status 404 returned error can't find the container with id c504ebe49ae6d2634d36ed2f6f3dcef4edb8cb82293411dad269b54228093a92 Jul 10 17:39:18.688876 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.688842 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:18.689150 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.688905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:18.689150 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.689026 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:39:18.689150 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.689029 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:39:18.689150 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.689041 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:18.689150 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.689085 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:39:18.689150 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.689041 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:39:18.689150 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.689105 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:19.689089001 +0000 UTC m=+37.516737748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:39:18.689150 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.689133 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:39:19.689119211 +0000 UTC m=+37.516767957 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:39:18.689435 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:18.689168 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:19.689154544 +0000 UTC m=+37.516803303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:39:18.750154 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.750072 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:18.750308 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.750181 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:18.750308 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.750181 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:18.753048 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.753028 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 17:39:18.753797 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.753777 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rxjk8\"" Jul 10 17:39:18.753938 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.753776 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 17:39:18.753938 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.753819 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 17:39:18.753938 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.753835 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n82vp\"" Jul 10 17:39:18.753938 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.753844 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 17:39:18.910478 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:18.910444 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dlx96" event={"ID":"c85d73b0-773f-4823-a71b-c8346bc8e11e","Type":"ContainerStarted","Data":"c504ebe49ae6d2634d36ed2f6f3dcef4edb8cb82293411dad269b54228093a92"} Jul 10 17:39:19.700178 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:19.700136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:19.700188 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:19.700286 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:19.700302 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:19.700325 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:19.700326 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:19.700391 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:21.700371443 +0000 UTC m=+39.528020191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:19.700407 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:39:21.700398631 +0000 UTC m=+39.528047378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:19.700409 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:39:19.700655 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:19.700501 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:21.700485275 +0000 UTC m=+39.528134042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:39:21.715952 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:21.715919 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:21.716005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:21.716038 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:21.716089 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:21.716156 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:25.716136132 +0000 UTC m=+43.543784899 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:21.716195 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:21.716248 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:39:25.716232032 +0000 UTC m=+43.543880778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:21.716316 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:21.716327 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:39:21.716473 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:21.716359 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:25.716349829 +0000 UTC m=+43.543998576 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:39:22.925335 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:22.925153 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dlx96" event={"ID":"c85d73b0-773f-4823-a71b-c8346bc8e11e","Type":"ContainerStarted","Data":"2314803087a75229f543b5c8b73341e6d5a9f1dd0297193b29931b43087b6da9"} Jul 10 17:39:22.928060 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:22.928030 2568 generic.go:356] "Generic (PLEG): container finished" podID="0108a2b0-af54-4d1b-804c-21f0fa2e8c61" containerID="78963a8a896f185b3faa280ac8de174bc3a04d8b68b908c6b90db45f0b472f36" exitCode=0 Jul 10 17:39:22.928135 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:22.928057 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" event={"ID":"0108a2b0-af54-4d1b-804c-21f0fa2e8c61","Type":"ContainerDied","Data":"78963a8a896f185b3faa280ac8de174bc3a04d8b68b908c6b90db45f0b472f36"} Jul 10 17:39:22.941588 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:22.941553 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dlx96" podStartSLOduration=2.460063812 podStartE2EDuration="5.941542135s" totalImagesPullingTime="3.481478323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:22.940638691 +0000 UTC m=+40.768287459" watchObservedRunningTime="2026-07-10 17:39:22.941542135 +0000 UTC m=+40.769190902" Jul 10 17:39:23.933191 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:23.933153 2568 generic.go:356] "Generic (PLEG): container finished" podID="0108a2b0-af54-4d1b-804c-21f0fa2e8c61" containerID="b8a9de6a421fbc0831911efc7bbcc384fe648ae4bf00ef30e9ab93de1a19c591" exitCode=0 Jul 10 17:39:23.933737 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:23.933229 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" event={"ID":"0108a2b0-af54-4d1b-804c-21f0fa2e8c61","Type":"ContainerDied","Data":"b8a9de6a421fbc0831911efc7bbcc384fe648ae4bf00ef30e9ab93de1a19c591"} Jul 10 17:39:24.938861 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:24.938827 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" event={"ID":"0108a2b0-af54-4d1b-804c-21f0fa2e8c61","Type":"ContainerStarted","Data":"ea5cc3a6c23965dd903193c7941cb620ade1a1da38ace08131875d5dcd98a61b"} Jul 10 17:39:24.963025 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:24.962956 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h2zbp" podStartSLOduration=8.550770858 podStartE2EDuration="41.96294025s" totalImagesPullingTime="33.412169392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:24.961364219 +0000 UTC m=+42.789012987" watchObservedRunningTime="2026-07-10 17:39:24.96294025 +0000 UTC m=+42.790589018" Jul 10 17:39:25.744764 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:25.744728 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:25.744764 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:25.744764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:25.744963 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:25.744811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:25.744963 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:25.744879 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:39:25.744963 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:25.744901 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:39:25.744963 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:25.744903 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:39:25.744963 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:25.744925 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:39:25.744963 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:25.744955 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:33.744941611 +0000 UTC m=+51.572590356 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:39:25.744963 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:25.744987 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:33.744961943 +0000 UTC m=+51.572610689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:39:25.745196 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:25.744998 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:39:33.744991933 +0000 UTC m=+51.572640679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:39:33.806703 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:33.806664 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:33.806703 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:33.806715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:33.807375 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:33.806793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:33.807375 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:33.806828 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:39:33.807375 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:33.806850 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:39:33.807375 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:33.806932 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:39:33.807375 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:33.806931 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:39:33.807375 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:33.806998 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:49.806962955 +0000 UTC m=+67.634611731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:39:33.807375 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:33.807020 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:39:49.807005845 +0000 UTC m=+67.634654591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:39:33.807375 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:33.807036 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:39:49.807028336 +0000 UTC m=+67.634677082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:39:44.929435 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:44.929404 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7hnvb" Jul 10 17:39:48.414833 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.414786 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:39:48.417915 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.417894 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 17:39:48.426019 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:48.425997 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 17:39:48.426107 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:48.426076 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:40:52.426054553 +0000 UTC m=+130.253703300 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : secret "metrics-daemon-secret" not found Jul 10 17:39:48.515937 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.515901 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:48.518481 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.518464 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 17:39:48.530787 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.530764 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/329c89c4-cfec-4be0-a5a8-aae54e2381b6-original-pull-secret\") pod \"global-pull-secret-syncer-9q7pt\" (UID: \"329c89c4-cfec-4be0-a5a8-aae54e2381b6\") " pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:48.616562 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.616538 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:48.619114 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.619095 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 17:39:48.629733 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.629713 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 17:39:48.640546 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.640525 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sqb9\" (UniqueName: \"kubernetes.io/projected/7119c54b-8e6a-4735-bc5e-6e64a41f6874-kube-api-access-6sqb9\") pod \"network-check-target-r29fc\" (UID: \"7119c54b-8e6a-4735-bc5e-6e64a41f6874\") " pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:48.766901 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.766824 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n82vp\"" Jul 10 17:39:48.770798 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.770783 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9q7pt" Jul 10 17:39:48.774807 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.774790 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:48.909713 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.909682 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9q7pt"] Jul 10 17:39:48.913724 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:39:48.913700 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod329c89c4_cfec_4be0_a5a8_aae54e2381b6.slice/crio-9621f01e46d52eb47ec130e6df6d07a1362020bc6b46604e970e6051ddc60586 WatchSource:0}: Error finding container 9621f01e46d52eb47ec130e6df6d07a1362020bc6b46604e970e6051ddc60586: Status 404 returned error can't find the container with id 9621f01e46d52eb47ec130e6df6d07a1362020bc6b46604e970e6051ddc60586 Jul 10 17:39:48.938849 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.938826 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r29fc"] Jul 10 17:39:48.941952 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:39:48.941932 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7119c54b_8e6a_4735_bc5e_6e64a41f6874.slice/crio-5fcd5edd994bdf048e16dc01620921ea5d0c4eed0161f5a0b843f7ffa97870c5 WatchSource:0}: Error finding container 5fcd5edd994bdf048e16dc01620921ea5d0c4eed0161f5a0b843f7ffa97870c5: Status 404 returned error can't find the container with id 5fcd5edd994bdf048e16dc01620921ea5d0c4eed0161f5a0b843f7ffa97870c5 Jul 10 17:39:48.995418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.995388 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r29fc" event={"ID":"7119c54b-8e6a-4735-bc5e-6e64a41f6874","Type":"ContainerStarted","Data":"5fcd5edd994bdf048e16dc01620921ea5d0c4eed0161f5a0b843f7ffa97870c5"} Jul 10 17:39:48.996403 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:48.996375 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9q7pt" event={"ID":"329c89c4-cfec-4be0-a5a8-aae54e2381b6","Type":"ContainerStarted","Data":"9621f01e46d52eb47ec130e6df6d07a1362020bc6b46604e970e6051ddc60586"} Jul 10 17:39:49.824689 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:49.824647 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:49.824720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:49.824753 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:49.824825 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:49.824876 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:49.824891 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:49.824892 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:49.824915 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:40:21.824892827 +0000 UTC m=+99.652541574 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:49.824936 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:40:21.824925626 +0000 UTC m=+99.652574372 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:39:49.825142 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:39:49.824948 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:40:21.824942409 +0000 UTC m=+99.652591154 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:39:54.011495 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:54.011401 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r29fc" event={"ID":"7119c54b-8e6a-4735-bc5e-6e64a41f6874","Type":"ContainerStarted","Data":"6d7411bd0441b46f76bf24897df8a32ebfa89b9359601055821131733ebac06c"} Jul 10 17:39:54.011960 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:54.011534 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:39:54.012909 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:54.012887 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9q7pt" event={"ID":"329c89c4-cfec-4be0-a5a8-aae54e2381b6","Type":"ContainerStarted","Data":"6b9c683b2c7eb573acdb61910a716488767fadfe453d181810541817e3685ee8"} Jul 10 17:39:54.028682 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:54.028641 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r29fc" podStartSLOduration=66.367855344 podStartE2EDuration="1m11.028629423s" totalImagesPullingTime="4.660774079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:54.027370243 +0000 UTC m=+71.855019033" watchObservedRunningTime="2026-07-10 17:39:54.028629423 +0000 UTC m=+71.856278190" Jul 10 17:39:54.041508 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:54.041463 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9q7pt" podStartSLOduration=66.311954381 podStartE2EDuration="1m11.041447487s" totalImagesPullingTime="4.729493106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:39:54.041139646 +0000 UTC m=+71.868788415" watchObservedRunningTime="2026-07-10 17:39:54.041447487 +0000 UTC m=+71.869096252" Jul 10 17:39:59.151042 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.151005 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh"] Jul 10 17:39:59.155028 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.155008 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.157400 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.157383 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 17:39:59.158446 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.158423 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 17:39:59.158545 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.158446 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 10 17:39:59.158545 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.158481 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 17:39:59.166191 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.166168 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh"] Jul 10 17:39:59.290193 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.290140 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/94cda4f5-487e-46b3-a08d-1f7c6647e2de-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.290440 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.290214 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94cda4f5-487e-46b3-a08d-1f7c6647e2de-tmp\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.290440 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.290240 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clcsn\" (UniqueName: \"kubernetes.io/projected/94cda4f5-487e-46b3-a08d-1f7c6647e2de-kube-api-access-clcsn\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.391656 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.391607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/94cda4f5-487e-46b3-a08d-1f7c6647e2de-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.391801 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.391678 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94cda4f5-487e-46b3-a08d-1f7c6647e2de-tmp\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.391801 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.391703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clcsn\" (UniqueName: \"kubernetes.io/projected/94cda4f5-487e-46b3-a08d-1f7c6647e2de-kube-api-access-clcsn\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.392166 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.392140 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/94cda4f5-487e-46b3-a08d-1f7c6647e2de-tmp\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.394355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.394338 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/94cda4f5-487e-46b3-a08d-1f7c6647e2de-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.399902 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.399880 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clcsn\" (UniqueName: \"kubernetes.io/projected/94cda4f5-487e-46b3-a08d-1f7c6647e2de-kube-api-access-clcsn\") pod \"klusterlet-addon-workmgr-6c85499b97-7n6qh\" (UID: \"94cda4f5-487e-46b3-a08d-1f7c6647e2de\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.464264 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.464192 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:39:59.582109 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:39:59.582080 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh"] Jul 10 17:39:59.585571 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:39:59.585539 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94cda4f5_487e_46b3_a08d_1f7c6647e2de.slice/crio-10de55842497a767209a864780ab4b9960051fee32739a18ce445669134d7bfe WatchSource:0}: Error finding container 10de55842497a767209a864780ab4b9960051fee32739a18ce445669134d7bfe: Status 404 returned error can't find the container with id 10de55842497a767209a864780ab4b9960051fee32739a18ce445669134d7bfe Jul 10 17:40:00.027849 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:00.027816 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" event={"ID":"94cda4f5-487e-46b3-a08d-1f7c6647e2de","Type":"ContainerStarted","Data":"10de55842497a767209a864780ab4b9960051fee32739a18ce445669134d7bfe"} Jul 10 17:40:04.039606 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:04.039497 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" event={"ID":"94cda4f5-487e-46b3-a08d-1f7c6647e2de","Type":"ContainerStarted","Data":"195eacc176fdd9bceffde459f1635b84c8de03aa062298ef283735b72c73ac67"} Jul 10 17:40:04.039986 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:04.039711 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:40:04.041390 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:04.041368 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:40:04.054710 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:04.054653 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" podStartSLOduration=1.565100111 podStartE2EDuration="5.054641297s" totalImagesPullingTime="3.489541186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:40:04.054577676 +0000 UTC m=+81.882226445" watchObservedRunningTime="2026-07-10 17:40:04.054641297 +0000 UTC m=+81.882290064" Jul 10 17:40:21.846348 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:21.846294 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:40:21.846348 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:21.846353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:40:21.846916 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:21.846450 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:40:21.846916 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:21.846450 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:40:21.846916 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:21.846583 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:40:21.846916 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:21.846597 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:40:21.846916 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:21.846623 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:41:25.846600083 +0000 UTC m=+163.674248830 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:40:21.846916 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:21.846648 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:41:25.846635616 +0000 UTC m=+163.674284381 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:40:21.846916 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:21.846667 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:40:21.847294 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:21.847108 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:41:25.84708619 +0000 UTC m=+163.674734952 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:40:25.017575 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:25.017541 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r29fc" Jul 10 17:40:52.483658 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:40:52.483595 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:40:52.484173 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:52.483754 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 17:40:52.484173 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:40:52.483818 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:42:54.4838025 +0000 UTC m=+252.311451249 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : secret "metrics-daemon-secret" not found Jul 10 17:41:20.997801 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:20.997759 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" podUID="b32a2e4f-6d0b-4933-b32c-3d14bcd50eee" Jul 10 17:41:21.006930 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:21.006904 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" podUID="848cca2f-0571-43b3-bef7-0c67b5b013f9" Jul 10 17:41:21.022201 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:21.022163 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7zqts" podUID="4b67c448-4305-4d2b-a46b-b87d9b3860b1" Jul 10 17:41:21.220029 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:41:21.219997 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:41:21.220207 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:41:21.219997 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7zqts" Jul 10 17:41:21.220207 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:41:21.220005 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:41:21.775657 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:21.775609 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:41:25.913850 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:41:25.913800 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:41:25.913871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:41:25.913903 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:25.913955 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:25.914032 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:25.914050 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-88d54bbc5-qgdnc: secret "image-registry-tls" not found Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:25.914051 2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:25.914042 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls podName:4b67c448-4305-4d2b-a46b-b87d9b3860b1 nodeName:}" failed. No retries permitted until 2026-07-10 17:43:27.914026582 +0000 UTC m=+285.741675328 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls") pod "dns-default-7zqts" (UID: "4b67c448-4305-4d2b-a46b-b87d9b3860b1") : secret "dns-default-metrics-tls" not found Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:25.914100 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls podName:848cca2f-0571-43b3-bef7-0c67b5b013f9 nodeName:}" failed. No retries permitted until 2026-07-10 17:43:27.914086802 +0000 UTC m=+285.741735551 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls") pod "image-registry-88d54bbc5-qgdnc" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9") : secret "image-registry-tls" not found Jul 10 17:41:25.914382 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:41:25.914111 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert podName:b32a2e4f-6d0b-4933-b32c-3d14bcd50eee nodeName:}" failed. No retries permitted until 2026-07-10 17:43:27.914105259 +0000 UTC m=+285.741754005 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-gl62k" (UID: "b32a2e4f-6d0b-4933-b32c-3d14bcd50eee") : secret "networking-console-plugin-cert" not found Jul 10 17:41:33.750783 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:41:33.750698 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:42:03.317050 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:03.317017 2568 generic.go:356] "Generic (PLEG): container finished" podID="94cda4f5-487e-46b3-a08d-1f7c6647e2de" containerID="195eacc176fdd9bceffde459f1635b84c8de03aa062298ef283735b72c73ac67" exitCode=1 Jul 10 17:42:03.317050 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:03.317052 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" event={"ID":"94cda4f5-487e-46b3-a08d-1f7c6647e2de","Type":"ContainerDied","Data":"195eacc176fdd9bceffde459f1635b84c8de03aa062298ef283735b72c73ac67"} Jul 10 17:42:03.317439 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:03.317319 2568 scope.go:117] "RemoveContainer" containerID="195eacc176fdd9bceffde459f1635b84c8de03aa062298ef283735b72c73ac67" Jul 10 17:42:04.039706 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:04.039656 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:42:04.323475 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:04.323385 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" event={"ID":"94cda4f5-487e-46b3-a08d-1f7c6647e2de","Type":"ContainerStarted","Data":"51ace99227375478fa7c1c3e3c4ec45f34a4a91d29ac40f9a3320a76f3d6632d"} Jul 10 17:42:04.323903 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:04.323581 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:42:04.324251 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:04.324238 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c85499b97-7n6qh" Jul 10 17:42:54.538432 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:54.538389 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:42:54.538915 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:42:54.538541 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 17:42:54.538915 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:42:54.538605 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs podName:d209f2ef-b7c8-4817-8950-548fbf79929d nodeName:}" failed. No retries permitted until 2026-07-10 17:44:56.538590678 +0000 UTC m=+374.366239424 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs") pod "network-metrics-daemon-l5nwq" (UID: "d209f2ef-b7c8-4817-8950-548fbf79929d") : secret "metrics-daemon-secret" not found Jul 10 17:42:59.675691 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.675655 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-77d769cf84-rmlf4"] Jul 10 17:42:59.678679 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.678656 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.680957 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.680924 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 17:42:59.681110 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.681028 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8bplx\"" Jul 10 17:42:59.681309 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.681293 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 17:42:59.681403 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.681294 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 17:42:59.681403 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.681326 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 17:42:59.681403 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.681400 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 17:42:59.681911 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.681897 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 17:42:59.696214 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.696191 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77d769cf84-rmlf4"] Jul 10 17:42:59.762569 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.762525 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-jxbvt"] Jul 10 17:42:59.765359 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.765344 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.768741 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.768718 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.768850 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.768803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-default-certificate\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.768850 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.768827 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-stats-auth\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.768942 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.768869 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78dzf\" (UniqueName: \"kubernetes.io/projected/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-kube-api-access-78dzf\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.768942 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.768914 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.769224 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.769204 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 17:42:59.769434 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.769418 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 17:42:59.769494 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.769434 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-db8b4\"" Jul 10 17:42:59.769494 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.769466 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 17:42:59.770067 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.770050 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 17:42:59.776474 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.776447 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 17:42:59.777679 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.777660 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-jxbvt"] Jul 10 17:42:59.870126 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870092 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wzh4\" (UniqueName: \"kubernetes.io/projected/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-kube-api-access-4wzh4\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.870126 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870128 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-snapshots\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.870325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870161 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.870325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.870325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870264 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-tmp\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.870325 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:42:59.870292 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:00.370275079 +0000 UTC m=+258.197923835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : configmap references non-existent config key: service-ca.crt Jul 10 17:42:59.870325 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:42:59.870315 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 17:42:59.870506 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:42:59.870369 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:00.370355696 +0000 UTC m=+258.198004442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : secret "router-metrics-certs-default" not found Jul 10 17:42:59.870506 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870318 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.870506 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-stats-auth\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.870506 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870425 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.870506 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870478 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-default-certificate\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.870691 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870509 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78dzf\" (UniqueName: \"kubernetes.io/projected/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-kube-api-access-78dzf\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.870691 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.870536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-serving-cert\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.872943 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.872917 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-default-certificate\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.873078 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.872947 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-stats-auth\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.878299 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.878280 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78dzf\" (UniqueName: \"kubernetes.io/projected/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-kube-api-access-78dzf\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:42:59.971486 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.971415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.971486 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.971455 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.971669 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.971508 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-serving-cert\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.971669 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.971525 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wzh4\" (UniqueName: \"kubernetes.io/projected/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-kube-api-access-4wzh4\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.971669 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.971543 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-snapshots\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.971669 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.971583 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-tmp\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.971963 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.971941 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-tmp\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.972126 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.972109 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.972285 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.972263 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-snapshots\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.972444 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.972430 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.973816 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.973795 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-serving-cert\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:42:59.979577 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:42:59.979558 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wzh4\" (UniqueName: \"kubernetes.io/projected/ea8f305d-b4f9-4c84-98b7-ce6ad74f3006-kube-api-access-4wzh4\") pod \"insights-operator-68d9dd94cf-jxbvt\" (UID: \"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006\") " pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:43:00.074615 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:00.074590 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" Jul 10 17:43:00.194436 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:00.194405 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-jxbvt"] Jul 10 17:43:00.197896 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:00.197870 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8f305d_b4f9_4c84_98b7_ce6ad74f3006.slice/crio-9600595671c85028c10a2ff0f6fa1157f75fc02cc3b007870cc350c79a3b509d WatchSource:0}: Error finding container 9600595671c85028c10a2ff0f6fa1157f75fc02cc3b007870cc350c79a3b509d: Status 404 returned error can't find the container with id 9600595671c85028c10a2ff0f6fa1157f75fc02cc3b007870cc350c79a3b509d Jul 10 17:43:00.375325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:00.375295 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:00.375515 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:00.375334 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:00.375515 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:00.375426 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 17:43:00.375515 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:00.375485 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:01.375461307 +0000 UTC m=+259.203110053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : configmap references non-existent config key: service-ca.crt Jul 10 17:43:00.375515 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:00.375514 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:01.375504316 +0000 UTC m=+259.203153062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : secret "router-metrics-certs-default" not found Jul 10 17:43:00.480227 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:00.480198 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" event={"ID":"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006","Type":"ContainerStarted","Data":"9600595671c85028c10a2ff0f6fa1157f75fc02cc3b007870cc350c79a3b509d"} Jul 10 17:43:01.381254 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:01.381216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:01.381729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:01.381276 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:01.381729 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:01.381423 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:03.381397931 +0000 UTC m=+261.209046689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : configmap references non-existent config key: service-ca.crt Jul 10 17:43:01.381729 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:01.381460 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 17:43:01.381729 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:01.381522 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:03.381502411 +0000 UTC m=+261.209151170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : secret "router-metrics-certs-default" not found Jul 10 17:43:02.491731 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:02.491684 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" event={"ID":"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006","Type":"ContainerStarted","Data":"ea8bb3da197f88b057ca6c18559389bd3407b15ba03880940ae5ba76fb9b6dc3"} Jul 10 17:43:02.509153 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:02.509102 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" podStartSLOduration=1.3103755750000001 podStartE2EDuration="3.509085466s" totalImagesPullingTime="2.198709891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:42:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:02.508483438 +0000 UTC m=+260.336132207" watchObservedRunningTime="2026-07-10 17:43:02.509085466 +0000 UTC m=+260.336734234" Jul 10 17:43:03.399710 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.399671 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:03.399880 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.399812 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:03.399880 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:03.399810 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 17:43:03.400034 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:03.399895 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:07.399875053 +0000 UTC m=+265.227523804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : secret "router-metrics-certs-default" not found Jul 10 17:43:03.400034 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:03.399912 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:07.399903293 +0000 UTC m=+265.227552045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : configmap references non-existent config key: service-ca.crt Jul 10 17:43:03.511852 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.511816 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cvxws"] Jul 10 17:43:03.515334 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.515309 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.517651 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.517624 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 17:43:03.517798 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.517754 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-66sjw\"" Jul 10 17:43:03.517861 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.517791 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 17:43:03.523166 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.523063 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cvxws"] Jul 10 17:43:03.601281 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.601203 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-data-volume\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.601281 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.601260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.601488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.601288 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4j2c\" (UniqueName: \"kubernetes.io/projected/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-api-access-l4j2c\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.601488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.601319 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-crio-socket\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.601594 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.601484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.702158 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.702088 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-data-volume\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.702158 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.702135 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.702361 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.702157 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4j2c\" (UniqueName: \"kubernetes.io/projected/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-api-access-l4j2c\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.702361 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.702190 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-crio-socket\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.702361 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:03.702218 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 17:43:03.702361 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.702269 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-crio-socket\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.702361 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:03.702273 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:04.202257839 +0000 UTC m=+262.029906588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : configmap "kube-rbac-proxy" not found Jul 10 17:43:03.702589 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.702380 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.702589 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.702436 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-data-volume\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:03.702589 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:03.702510 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 17:43:03.702589 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:03.702574 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:04.202555671 +0000 UTC m=+262.030204437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : secret "insights-runtime-extractor-tls" not found Jul 10 17:43:03.713028 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:03.712998 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4j2c\" (UniqueName: \"kubernetes.io/projected/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-api-access-l4j2c\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:04.207493 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:04.207451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:04.207667 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:04.207503 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:04.207667 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:04.207573 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 17:43:04.207667 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:04.207588 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 17:43:04.207667 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:04.207651 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:05.207629112 +0000 UTC m=+263.035277860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : secret "insights-runtime-extractor-tls" not found Jul 10 17:43:04.207806 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:04.207691 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:05.207679226 +0000 UTC m=+263.035327973 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : configmap "kube-rbac-proxy" not found Jul 10 17:43:05.215946 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:05.215904 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:05.216398 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:05.216069 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 17:43:05.216398 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:05.216070 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:05.216398 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:05.216131 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 17:43:05.216398 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:05.216135 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:07.216121397 +0000 UTC m=+265.043770143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : secret "insights-runtime-extractor-tls" not found Jul 10 17:43:05.216398 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:05.216214 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:07.216199116 +0000 UTC m=+265.043847865 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : configmap "kube-rbac-proxy" not found Jul 10 17:43:05.710109 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:05.710084 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lklvj_057ec1b6-2d2e-4a4b-baac-ddd4f33b302a/dns-node-resolver/0.log" Jul 10 17:43:06.710624 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:06.710574 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zg66n_ca928c87-0863-459c-bff1-2c7d3d1dc864/node-ca/0.log" Jul 10 17:43:07.113239 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:07.113212 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dlx96_c85d73b0-773f-4823-a71b-c8346bc8e11e/serve-healthcheck-canary/0.log" Jul 10 17:43:07.235582 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:07.232486 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:07.235582 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:07.232561 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:07.235582 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:07.232729 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 17:43:07.235582 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:07.232804 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:11.23277966 +0000 UTC m=+269.060428408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : secret "insights-runtime-extractor-tls" not found Jul 10 17:43:07.235582 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:07.233277 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 17:43:07.235582 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:07.233334 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:11.233310674 +0000 UTC m=+269.060959428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : configmap "kube-rbac-proxy" not found Jul 10 17:43:07.433761 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:07.433670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:07.433761 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:07.433712 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:07.433994 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:07.433821 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 17:43:07.433994 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:07.433890 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:15.43386667 +0000 UTC m=+273.261515417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : configmap references non-existent config key: service-ca.crt Jul 10 17:43:07.433994 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:07.433942 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs podName:8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c nodeName:}" failed. No retries permitted until 2026-07-10 17:43:15.433929848 +0000 UTC m=+273.261578594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs") pod "router-default-77d769cf84-rmlf4" (UID: "8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c") : secret "router-metrics-certs-default" not found Jul 10 17:43:11.264693 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:11.264652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:11.264693 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:11.264708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:11.265172 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:11.264782 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 17:43:11.265172 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:11.264800 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 17:43:11.265172 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:11.264862 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:19.264842566 +0000 UTC m=+277.092491325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : configmap "kube-rbac-proxy" not found Jul 10 17:43:11.265172 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:11.264922 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls podName:531b0c4e-b6d4-4e2d-bb1a-d36e8572792e nodeName:}" failed. No retries permitted until 2026-07-10 17:43:19.26491503 +0000 UTC m=+277.092563778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cvxws" (UID: "531b0c4e-b6d4-4e2d-bb1a-d36e8572792e") : secret "insights-runtime-extractor-tls" not found Jul 10 17:43:15.498616 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:15.498015 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:15.498616 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:15.498090 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:15.502065 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:15.502035 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-metrics-certs\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:15.502218 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:15.502197 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c-service-ca-bundle\") pod \"router-default-77d769cf84-rmlf4\" (UID: \"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c\") " pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:15.586998 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:15.586956 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:15.711845 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:15.711819 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77d769cf84-rmlf4"] Jul 10 17:43:15.713855 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:15.713832 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7f23a3_7a79_4e3a_9e23_3dc9132fa80c.slice/crio-fbea4a60865b6ab2dc0f4ceed69705764cf2d1cf54d71184ed69904f70fadf2b WatchSource:0}: Error finding container fbea4a60865b6ab2dc0f4ceed69705764cf2d1cf54d71184ed69904f70fadf2b: Status 404 returned error can't find the container with id fbea4a60865b6ab2dc0f4ceed69705764cf2d1cf54d71184ed69904f70fadf2b Jul 10 17:43:16.532422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:16.532388 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77d769cf84-rmlf4" event={"ID":"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c","Type":"ContainerStarted","Data":"8547b34356487a00054425579206a19618bfac4c9d456b878b9e11d5c92fd6c0"} Jul 10 17:43:16.532792 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:16.532435 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77d769cf84-rmlf4" event={"ID":"8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c","Type":"ContainerStarted","Data":"fbea4a60865b6ab2dc0f4ceed69705764cf2d1cf54d71184ed69904f70fadf2b"} Jul 10 17:43:16.550629 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:16.550577 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-77d769cf84-rmlf4" podStartSLOduration=17.550561057 podStartE2EDuration="17.550561057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:16.54980005 +0000 UTC m=+274.377448819" watchObservedRunningTime="2026-07-10 17:43:16.550561057 +0000 UTC m=+274.378209802" Jul 10 17:43:16.587858 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:16.587832 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:16.590300 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:16.590282 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:17.535718 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:17.535691 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:17.536882 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:17.536861 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-77d769cf84-rmlf4" Jul 10 17:43:19.325110 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:19.325063 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:19.325509 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:19.325142 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:19.325649 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:19.325630 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:19.327450 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:19.327432 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/531b0c4e-b6d4-4e2d-bb1a-d36e8572792e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cvxws\" (UID: \"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e\") " pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:19.426936 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:19.426909 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cvxws" Jul 10 17:43:19.550134 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:19.550105 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cvxws"] Jul 10 17:43:19.552934 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:19.552903 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531b0c4e_b6d4_4e2d_bb1a_d36e8572792e.slice/crio-17e7e68fa6bbc8a4c3f1ffa233ab5f255057df125634bcb9689b8a0ee483183e WatchSource:0}: Error finding container 17e7e68fa6bbc8a4c3f1ffa233ab5f255057df125634bcb9689b8a0ee483183e: Status 404 returned error can't find the container with id 17e7e68fa6bbc8a4c3f1ffa233ab5f255057df125634bcb9689b8a0ee483183e Jul 10 17:43:20.544841 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:20.544802 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cvxws" event={"ID":"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e","Type":"ContainerStarted","Data":"17e7e68fa6bbc8a4c3f1ffa233ab5f255057df125634bcb9689b8a0ee483183e"} Jul 10 17:43:23.555271 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:23.555187 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cvxws" event={"ID":"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e","Type":"ContainerStarted","Data":"57d1d0ca56468eab7be25a70cc1822fa2186384de87782b86e911773a5ec1e60"} Jul 10 17:43:24.220908 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:24.220870 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" podUID="b32a2e4f-6d0b-4933-b32c-3d14bcd50eee" Jul 10 17:43:24.220908 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:24.220869 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7zqts" podUID="4b67c448-4305-4d2b-a46b-b87d9b3860b1" Jul 10 17:43:24.220908 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:24.220869 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" podUID="848cca2f-0571-43b3-bef7-0c67b5b013f9" Jul 10 17:43:24.560242 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:24.560146 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:43:24.560242 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:24.560186 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:43:24.560242 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:24.560203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cvxws" event={"ID":"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e","Type":"ContainerStarted","Data":"5a0c5f7c001b4fcb8979f205d6600e515ee000f1ca87db611c24fe15a3085bf2"} Jul 10 17:43:24.560753 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:24.560265 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7zqts" Jul 10 17:43:26.571255 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:26.571215 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cvxws" event={"ID":"531b0c4e-b6d4-4e2d-bb1a-d36e8572792e","Type":"ContainerStarted","Data":"d47e739b2cbf9360d82504d321353cff6377b60800bd4fcef522e8bbe88c26d7"} Jul 10 17:43:26.588281 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:26.588195 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cvxws" podStartSLOduration=16.926325603 podStartE2EDuration="23.588182333s" totalImagesPullingTime="6.66185673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:43:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:26.587479824 +0000 UTC m=+284.415128592" watchObservedRunningTime="2026-07-10 17:43:26.588182333 +0000 UTC m=+284.415831100" Jul 10 17:43:27.640056 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.640025 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6"] Jul 10 17:43:27.643288 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.643267 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" Jul 10 17:43:27.645994 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.645952 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 10 17:43:27.646985 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.646949 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8jmfc\"" Jul 10 17:43:27.668383 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.668362 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6"] Jul 10 17:43:27.679056 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.679036 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68dd868d65-kkzvz"] Jul 10 17:43:27.681870 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.681856 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-g86hh"] Jul 10 17:43:27.682013 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.681997 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.684726 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.684705 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-g86hh" Jul 10 17:43:27.685611 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.685589 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 17:43:27.685801 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.685773 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-z6jhd\"" Jul 10 17:43:27.685886 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.685681 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 17:43:27.685955 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.685681 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 17:43:27.686167 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.686146 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 17:43:27.686253 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.686180 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 17:43:27.686378 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.686355 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 17:43:27.686695 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.686678 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 17:43:27.689545 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.689525 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qlv59\"" Jul 10 17:43:27.699472 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.699448 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-g86hh"] Jul 10 17:43:27.700747 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.700728 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68dd868d65-kkzvz"] Jul 10 17:43:27.800730 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.800699 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-service-ca\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.800879 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.800734 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-config\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.800879 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.800775 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/565eecbc-a422-4a3c-9e14-9b660e2a938d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-czkw6\" (UID: \"565eecbc-a422-4a3c-9e14-9b660e2a938d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" Jul 10 17:43:27.800879 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.800796 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjq9n\" (UniqueName: \"kubernetes.io/projected/8f558d18-fa8d-4b98-8610-2427c33f7a4a-kube-api-access-hjq9n\") pod \"downloads-68c9cbb497-g86hh\" (UID: \"8f558d18-fa8d-4b98-8610-2427c33f7a4a\") " pod="openshift-console/downloads-68c9cbb497-g86hh" Jul 10 17:43:27.800879 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.800827 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-oauth-config\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.801118 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.800874 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vxhp\" (UniqueName: \"kubernetes.io/projected/61eb2013-8a8d-4e81-adff-5ff31f6a766e-kube-api-access-9vxhp\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.801118 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.800951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-serving-cert\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.801118 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.801006 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-oauth-serving-cert\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.902033 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.901944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-service-ca\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.902033 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.901992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-config\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.902033 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.902029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/565eecbc-a422-4a3c-9e14-9b660e2a938d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-czkw6\" (UID: \"565eecbc-a422-4a3c-9e14-9b660e2a938d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" Jul 10 17:43:27.902241 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.902049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjq9n\" (UniqueName: \"kubernetes.io/projected/8f558d18-fa8d-4b98-8610-2427c33f7a4a-kube-api-access-hjq9n\") pod \"downloads-68c9cbb497-g86hh\" (UID: \"8f558d18-fa8d-4b98-8610-2427c33f7a4a\") " pod="openshift-console/downloads-68c9cbb497-g86hh" Jul 10 17:43:27.902241 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.902073 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-oauth-config\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.902241 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.902089 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxhp\" (UniqueName: \"kubernetes.io/projected/61eb2013-8a8d-4e81-adff-5ff31f6a766e-kube-api-access-9vxhp\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.902385 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.902266 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-serving-cert\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.902521 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.902492 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-oauth-serving-cert\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.902934 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.902753 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-config\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.902934 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.902731 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-service-ca\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.903113 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.903095 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-oauth-serving-cert\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.904666 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.904637 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-oauth-config\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.904666 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.904659 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/565eecbc-a422-4a3c-9e14-9b660e2a938d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-czkw6\" (UID: \"565eecbc-a422-4a3c-9e14-9b660e2a938d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" Jul 10 17:43:27.904791 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.904720 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-serving-cert\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.913603 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.913579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjq9n\" (UniqueName: \"kubernetes.io/projected/8f558d18-fa8d-4b98-8610-2427c33f7a4a-kube-api-access-hjq9n\") pod \"downloads-68c9cbb497-g86hh\" (UID: \"8f558d18-fa8d-4b98-8610-2427c33f7a4a\") " pod="openshift-console/downloads-68c9cbb497-g86hh" Jul 10 17:43:27.915109 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.915089 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vxhp\" (UniqueName: \"kubernetes.io/projected/61eb2013-8a8d-4e81-adff-5ff31f6a766e-kube-api-access-9vxhp\") pod \"console-68dd868d65-kkzvz\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:27.951822 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.951800 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" Jul 10 17:43:27.993908 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:27.993881 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:28.000664 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.000057 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-g86hh" Jul 10 17:43:28.003638 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.003610 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:43:28.003749 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.003665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:43:28.003749 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.003700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:43:28.007220 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.007170 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b32a2e4f-6d0b-4933-b32c-3d14bcd50eee-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-gl62k\" (UID: \"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:43:28.007330 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.007318 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"image-registry-88d54bbc5-qgdnc\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:43:28.007789 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.007764 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b67c448-4305-4d2b-a46b-b87d9b3860b1-metrics-tls\") pod \"dns-default-7zqts\" (UID: \"4b67c448-4305-4d2b-a46b-b87d9b3860b1\") " pod="openshift-dns/dns-default-7zqts" Jul 10 17:43:28.094230 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.094196 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6"] Jul 10 17:43:28.097552 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:28.097526 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod565eecbc_a422_4a3c_9e14_9b660e2a938d.slice/crio-2e0d352ec67c7cd569a3f0b3eb7f8b36c87d42e35fca4fae86375a1a2472d2b8 WatchSource:0}: Error finding container 2e0d352ec67c7cd569a3f0b3eb7f8b36c87d42e35fca4fae86375a1a2472d2b8: Status 404 returned error can't find the container with id 2e0d352ec67c7cd569a3f0b3eb7f8b36c87d42e35fca4fae86375a1a2472d2b8 Jul 10 17:43:28.141696 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.141668 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68dd868d65-kkzvz"] Jul 10 17:43:28.144298 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:28.144266 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61eb2013_8a8d_4e81_adff_5ff31f6a766e.slice/crio-f2109f1ab26ac330ccf3afc74836277653a5b7751526f45bbb588136aff69988 WatchSource:0}: Error finding container f2109f1ab26ac330ccf3afc74836277653a5b7751526f45bbb588136aff69988: Status 404 returned error can't find the container with id f2109f1ab26ac330ccf3afc74836277653a5b7751526f45bbb588136aff69988 Jul 10 17:43:28.157874 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.157818 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-g86hh"] Jul 10 17:43:28.161028 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:28.161001 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f558d18_fa8d_4b98_8610_2427c33f7a4a.slice/crio-0e1cf4e4b50de73038466c246f183ab1e619c0a2a3b02ac700bd45c171427a4b WatchSource:0}: Error finding container 0e1cf4e4b50de73038466c246f183ab1e619c0a2a3b02ac700bd45c171427a4b: Status 404 returned error can't find the container with id 0e1cf4e4b50de73038466c246f183ab1e619c0a2a3b02ac700bd45c171427a4b Jul 10 17:43:28.163279 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.163159 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-z8xbl\"" Jul 10 17:43:28.163279 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.163201 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tc8df\"" Jul 10 17:43:28.163279 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.163255 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dbkj9\"" Jul 10 17:43:28.171757 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.171728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:43:28.171757 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.171738 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" Jul 10 17:43:28.171887 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.171763 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7zqts" Jul 10 17:43:28.314026 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.313990 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-gl62k"] Jul 10 17:43:28.317537 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:28.317507 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32a2e4f_6d0b_4933_b32c_3d14bcd50eee.slice/crio-cfa8d8af4288372c9beea2342123b55a07f4238e3e08fe2736604d5e823da82b WatchSource:0}: Error finding container cfa8d8af4288372c9beea2342123b55a07f4238e3e08fe2736604d5e823da82b: Status 404 returned error can't find the container with id cfa8d8af4288372c9beea2342123b55a07f4238e3e08fe2736604d5e823da82b Jul 10 17:43:28.538001 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.537908 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7zqts"] Jul 10 17:43:28.540819 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.540797 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-88d54bbc5-qgdnc"] Jul 10 17:43:28.542276 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:28.542255 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b67c448_4305_4d2b_a46b_b87d9b3860b1.slice/crio-763f448003c3c5651c3e6d55efa89c9d2442b615759a13dffd9ca02894409dca WatchSource:0}: Error finding container 763f448003c3c5651c3e6d55efa89c9d2442b615759a13dffd9ca02894409dca: Status 404 returned error can't find the container with id 763f448003c3c5651c3e6d55efa89c9d2442b615759a13dffd9ca02894409dca Jul 10 17:43:28.543705 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:28.543673 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848cca2f_0571_43b3_bef7_0c67b5b013f9.slice/crio-501fc13552e0c996eccae2f332fb029528c07c9db4b3ba4e0bc061c3c0b6b4f4 WatchSource:0}: Error finding container 501fc13552e0c996eccae2f332fb029528c07c9db4b3ba4e0bc061c3c0b6b4f4: Status 404 returned error can't find the container with id 501fc13552e0c996eccae2f332fb029528c07c9db4b3ba4e0bc061c3c0b6b4f4 Jul 10 17:43:28.578671 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.578640 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zqts" event={"ID":"4b67c448-4305-4d2b-a46b-b87d9b3860b1","Type":"ContainerStarted","Data":"763f448003c3c5651c3e6d55efa89c9d2442b615759a13dffd9ca02894409dca"} Jul 10 17:43:28.579839 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.579815 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-g86hh" event={"ID":"8f558d18-fa8d-4b98-8610-2427c33f7a4a","Type":"ContainerStarted","Data":"0e1cf4e4b50de73038466c246f183ab1e619c0a2a3b02ac700bd45c171427a4b"} Jul 10 17:43:28.580879 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.580858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" event={"ID":"565eecbc-a422-4a3c-9e14-9b660e2a938d","Type":"ContainerStarted","Data":"2e0d352ec67c7cd569a3f0b3eb7f8b36c87d42e35fca4fae86375a1a2472d2b8"} Jul 10 17:43:28.581941 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.581912 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" event={"ID":"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee","Type":"ContainerStarted","Data":"cfa8d8af4288372c9beea2342123b55a07f4238e3e08fe2736604d5e823da82b"} Jul 10 17:43:28.583086 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.583064 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" event={"ID":"848cca2f-0571-43b3-bef7-0c67b5b013f9","Type":"ContainerStarted","Data":"501fc13552e0c996eccae2f332fb029528c07c9db4b3ba4e0bc061c3c0b6b4f4"} Jul 10 17:43:28.584264 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:28.584243 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68dd868d65-kkzvz" event={"ID":"61eb2013-8a8d-4e81-adff-5ff31f6a766e","Type":"ContainerStarted","Data":"f2109f1ab26ac330ccf3afc74836277653a5b7751526f45bbb588136aff69988"} Jul 10 17:43:29.595019 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:29.594955 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" event={"ID":"848cca2f-0571-43b3-bef7-0c67b5b013f9","Type":"ContainerStarted","Data":"6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348"} Jul 10 17:43:29.595490 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:29.595111 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:43:29.619449 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:29.619381 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" podStartSLOduration=286.619360505 podStartE2EDuration="4m46.619360505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:29.617610351 +0000 UTC m=+287.445259119" watchObservedRunningTime="2026-07-10 17:43:29.619360505 +0000 UTC m=+287.447009259" Jul 10 17:43:30.600844 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:30.600803 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" event={"ID":"565eecbc-a422-4a3c-9e14-9b660e2a938d","Type":"ContainerStarted","Data":"c6e871cae1f8dd21866d8eec8053fd5fd2d74b96eb1a7c55c4089b70ae64e323"} Jul 10 17:43:30.601349 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:30.601079 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" Jul 10 17:43:30.604317 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:30.604272 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" event={"ID":"b32a2e4f-6d0b-4933-b32c-3d14bcd50eee","Type":"ContainerStarted","Data":"b0cb5d4b86f3aa7827dbe8018f4a5cdc45db7300097e9c6f0da2e855cefd0050"} Jul 10 17:43:30.607315 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:30.607285 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" Jul 10 17:43:30.619541 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:30.619494 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-czkw6" podStartSLOduration=1.670297808 podStartE2EDuration="3.619474626s" totalImagesPullingTime="1.949176818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:30.618359051 +0000 UTC m=+288.446007823" watchObservedRunningTime="2026-07-10 17:43:30.619474626 +0000 UTC m=+288.447123395" Jul 10 17:43:30.636686 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:30.636586 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-gl62k" podStartSLOduration=272.904900347 podStartE2EDuration="4m34.636567021s" totalImagesPullingTime="1.731666674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:38:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:30.635142345 +0000 UTC m=+288.462791115" watchObservedRunningTime="2026-07-10 17:43:30.636567021 +0000 UTC m=+288.464215789" Jul 10 17:43:31.731767 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.731731 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-9http"] Jul 10 17:43:31.736637 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.736612 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.739299 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.739273 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 17:43:31.739541 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.739523 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 17:43:31.739753 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.739734 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 17:43:31.740580 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.740417 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 17:43:31.740580 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.740525 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qkvcl\"" Jul 10 17:43:31.740580 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.740556 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 17:43:31.745025 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.744910 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-9http"] Jul 10 17:43:31.840636 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.840595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-metrics-client-ca\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.840813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.840651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7qpp\" (UniqueName: \"kubernetes.io/projected/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-kube-api-access-q7qpp\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.840813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.840752 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.840927 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.840832 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.941656 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.941621 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-metrics-client-ca\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.941828 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.941677 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7qpp\" (UniqueName: \"kubernetes.io/projected/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-kube-api-access-q7qpp\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.941828 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.941729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.941828 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.941788 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.942514 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.942476 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-metrics-client-ca\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.945393 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.945320 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.945393 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.945349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:31.950425 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:31.950366 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7qpp\" (UniqueName: \"kubernetes.io/projected/3dfe8a3d-13a6-430b-9cb7-6000c7b00827-kube-api-access-q7qpp\") pod \"prometheus-operator-64785b6675-9http\" (UID: \"3dfe8a3d-13a6-430b-9cb7-6000c7b00827\") " pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:32.049816 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.049724 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-9http" Jul 10 17:43:32.481072 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.480952 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-9http"] Jul 10 17:43:32.484452 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:32.484369 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dfe8a3d_13a6_430b_9cb7_6000c7b00827.slice/crio-d55fc45ee492c22f3d1055ec99d8dbbae6127999de70a9ee26e4423b780fdfd8 WatchSource:0}: Error finding container d55fc45ee492c22f3d1055ec99d8dbbae6127999de70a9ee26e4423b780fdfd8: Status 404 returned error can't find the container with id d55fc45ee492c22f3d1055ec99d8dbbae6127999de70a9ee26e4423b780fdfd8 Jul 10 17:43:32.613365 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.613331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zqts" event={"ID":"4b67c448-4305-4d2b-a46b-b87d9b3860b1","Type":"ContainerStarted","Data":"971c1cba469f7a7fcf62834a42657278ae69055c809ceb1bc17f5c636e7a8dff"} Jul 10 17:43:32.613529 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.613390 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zqts" event={"ID":"4b67c448-4305-4d2b-a46b-b87d9b3860b1","Type":"ContainerStarted","Data":"6adf3c3061ffb65c12b06e8f181fdb4757bd22c4edcab3368a78f56367b98964"} Jul 10 17:43:32.613607 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.613560 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7zqts" Jul 10 17:43:32.614665 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.614635 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-9http" event={"ID":"3dfe8a3d-13a6-430b-9cb7-6000c7b00827","Type":"ContainerStarted","Data":"d55fc45ee492c22f3d1055ec99d8dbbae6127999de70a9ee26e4423b780fdfd8"} Jul 10 17:43:32.616601 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.616575 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68dd868d65-kkzvz" event={"ID":"61eb2013-8a8d-4e81-adff-5ff31f6a766e","Type":"ContainerStarted","Data":"733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268"} Jul 10 17:43:32.630399 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.630350 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7zqts" podStartSLOduration=251.855369855 podStartE2EDuration="4m15.630330763s" totalImagesPullingTime="3.774960908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:32.628296585 +0000 UTC m=+290.455945354" watchObservedRunningTime="2026-07-10 17:43:32.630330763 +0000 UTC m=+290.457979532" Jul 10 17:43:32.644987 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:32.644923 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68dd868d65-kkzvz" podStartSLOduration=1.46459342 podStartE2EDuration="5.644905478s" totalImagesPullingTime="4.180312058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:32.644043739 +0000 UTC m=+290.471692508" watchObservedRunningTime="2026-07-10 17:43:32.644905478 +0000 UTC m=+290.472554245" Jul 10 17:43:34.626925 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:34.626883 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-9http" event={"ID":"3dfe8a3d-13a6-430b-9cb7-6000c7b00827","Type":"ContainerStarted","Data":"6222338e77305b2b42fa1b9ba09c9edc97da09724afe7f9401868c7c4a77547b"} Jul 10 17:43:34.627342 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:34.626952 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-9http" event={"ID":"3dfe8a3d-13a6-430b-9cb7-6000c7b00827","Type":"ContainerStarted","Data":"382d9b4a30162cbf0642696bb5c083af28432f5f3a935df6debe5c75463e48f5"} Jul 10 17:43:34.644346 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:34.644300 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-9http" podStartSLOduration=2.212984712 podStartE2EDuration="3.644284347s" totalImagesPullingTime="1.431299635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:34.642464251 +0000 UTC m=+292.470113020" watchObservedRunningTime="2026-07-10 17:43:34.644284347 +0000 UTC m=+292.471933118" Jul 10 17:43:36.100325 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.100296 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h6p5c"] Jul 10 17:43:36.105562 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.105540 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.108397 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.108368 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 17:43:36.108532 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.108419 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 17:43:36.108532 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.108428 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 17:43:36.108739 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.108720 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lp7qm\"" Jul 10 17:43:36.177081 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-textfile\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.177081 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177085 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-wtmp\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.177304 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177172 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-sys\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.177304 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177197 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.177304 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177226 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7n6c\" (UniqueName: \"kubernetes.io/projected/946fb184-bf08-4f31-8ae3-37e1d2891cba-kube-api-access-f7n6c\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.177445 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177312 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.177445 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177349 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/946fb184-bf08-4f31-8ae3-37e1d2891cba-metrics-client-ca\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.177445 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-tls\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.177445 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.177426 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-root\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278338 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278304 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-sys\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278508 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278348 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278508 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278381 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7n6c\" (UniqueName: \"kubernetes.io/projected/946fb184-bf08-4f31-8ae3-37e1d2891cba-kube-api-access-f7n6c\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278508 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278508 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278435 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/946fb184-bf08-4f31-8ae3-37e1d2891cba-metrics-client-ca\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278508 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278472 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-tls\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278508 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-root\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278828 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278521 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-textfile\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278828 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278544 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-wtmp\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278828 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278409 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-sys\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.278828 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:36.278645 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 17:43:36.278828 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:36.278725 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-tls podName:946fb184-bf08-4f31-8ae3-37e1d2891cba nodeName:}" failed. No retries permitted until 2026-07-10 17:43:36.778704752 +0000 UTC m=+294.606353501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-tls") pod "node-exporter-h6p5c" (UID: "946fb184-bf08-4f31-8ae3-37e1d2891cba") : secret "node-exporter-tls" not found Jul 10 17:43:36.279119 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278844 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-wtmp\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.279119 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.278626 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/946fb184-bf08-4f31-8ae3-37e1d2891cba-root\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.279119 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.279049 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.279249 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.279142 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/946fb184-bf08-4f31-8ae3-37e1d2891cba-metrics-client-ca\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.279249 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.279204 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-textfile\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.281136 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.281113 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.288898 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.288879 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7n6c\" (UniqueName: \"kubernetes.io/projected/946fb184-bf08-4f31-8ae3-37e1d2891cba-kube-api-access-f7n6c\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.752037 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:43:36.751996 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-l5nwq" podUID="d209f2ef-b7c8-4817-8950-548fbf79929d" Jul 10 17:43:36.782150 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.782110 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-tls\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:36.784950 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:36.784921 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/946fb184-bf08-4f31-8ae3-37e1d2891cba-node-exporter-tls\") pod \"node-exporter-h6p5c\" (UID: \"946fb184-bf08-4f31-8ae3-37e1d2891cba\") " pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:37.015094 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:37.014994 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6p5c" Jul 10 17:43:37.031992 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:37.031935 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946fb184_bf08_4f31_8ae3_37e1d2891cba.slice/crio-3f031dec41bea3e84a6b1c001591b59b6b35f827b9d97e0cb827d76c494342d7 WatchSource:0}: Error finding container 3f031dec41bea3e84a6b1c001591b59b6b35f827b9d97e0cb827d76c494342d7: Status 404 returned error can't find the container with id 3f031dec41bea3e84a6b1c001591b59b6b35f827b9d97e0cb827d76c494342d7 Jul 10 17:43:37.644324 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:37.644284 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6p5c" event={"ID":"946fb184-bf08-4f31-8ae3-37e1d2891cba","Type":"ContainerStarted","Data":"3f031dec41bea3e84a6b1c001591b59b6b35f827b9d97e0cb827d76c494342d7"} Jul 10 17:43:37.994580 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:37.994499 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:37.994580 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:37.994562 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:43:37.996282 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:37.996250 2568 patch_prober.go:28] interesting pod/console-68dd868d65-kkzvz container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.15:8443/health\": dial tcp 10.132.0.15:8443: connect: connection refused" start-of-body= Jul 10 17:43:37.996419 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:37.996304 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-68dd868d65-kkzvz" podUID="61eb2013-8a8d-4e81-adff-5ff31f6a766e" containerName="console" probeResult="failure" output="Get \"https://10.132.0.15:8443/health\": dial tcp 10.132.0.15:8443: connect: connection refused" Jul 10 17:43:38.066937 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.066908 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-655b67cf94-vb9rq"] Jul 10 17:43:38.070767 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.070736 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.073624 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.073597 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 10 17:43:38.073756 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.073673 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 10 17:43:38.073756 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.073730 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 10 17:43:38.073878 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.073730 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-aktvk5g00gfep\"" Jul 10 17:43:38.074229 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.074129 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 10 17:43:38.074229 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.074162 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j6gfq\"" Jul 10 17:43:38.074229 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.074202 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 10 17:43:38.089807 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.089783 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-655b67cf94-vb9rq"] Jul 10 17:43:38.094368 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.094345 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.094476 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.094398 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.094557 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.094536 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-grpc-tls\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.094724 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.094700 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.094831 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.094775 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.094831 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.094810 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10d50cd8-583f-4b31-9667-4079d4a71a72-metrics-client-ca\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.094905 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.094847 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwlht\" (UniqueName: \"kubernetes.io/projected/10d50cd8-583f-4b31-9667-4079d4a71a72-kube-api-access-fwlht\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.094952 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.094935 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-tls\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.196155 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.196132 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwlht\" (UniqueName: \"kubernetes.io/projected/10d50cd8-583f-4b31-9667-4079d4a71a72-kube-api-access-fwlht\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.196286 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.196183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-tls\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.196286 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.196216 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.196286 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.196244 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.196286 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.196272 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-grpc-tls\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.196552 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.196337 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.196552 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.196368 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.196659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.196547 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10d50cd8-583f-4b31-9667-4079d4a71a72-metrics-client-ca\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.197560 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.197480 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10d50cd8-583f-4b31-9667-4079d4a71a72-metrics-client-ca\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.199567 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.199507 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.199955 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.199885 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.200364 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.200336 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-tls\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.200902 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.200860 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.201148 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.201098 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-grpc-tls\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.201864 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.201838 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/10d50cd8-583f-4b31-9667-4079d4a71a72-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.205997 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.205956 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwlht\" (UniqueName: \"kubernetes.io/projected/10d50cd8-583f-4b31-9667-4079d4a71a72-kube-api-access-fwlht\") pod \"thanos-querier-655b67cf94-vb9rq\" (UID: \"10d50cd8-583f-4b31-9667-4079d4a71a72\") " pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.381600 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.381567 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:38.520292 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.520260 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-655b67cf94-vb9rq"] Jul 10 17:43:38.523554 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:38.523525 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d50cd8_583f_4b31_9667_4079d4a71a72.slice/crio-9a0441839feb71cd9bcc9c88e8882460574b9f9310ef8eef975291c4f8a6740e WatchSource:0}: Error finding container 9a0441839feb71cd9bcc9c88e8882460574b9f9310ef8eef975291c4f8a6740e: Status 404 returned error can't find the container with id 9a0441839feb71cd9bcc9c88e8882460574b9f9310ef8eef975291c4f8a6740e Jul 10 17:43:38.650522 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.650474 2568 generic.go:356] "Generic (PLEG): container finished" podID="946fb184-bf08-4f31-8ae3-37e1d2891cba" containerID="f5ebd40d5eed168bafdf7caceede2f8b4834d078873e81afc1206ba6ce7759e2" exitCode=0 Jul 10 17:43:38.650959 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.650574 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6p5c" event={"ID":"946fb184-bf08-4f31-8ae3-37e1d2891cba","Type":"ContainerDied","Data":"f5ebd40d5eed168bafdf7caceede2f8b4834d078873e81afc1206ba6ce7759e2"} Jul 10 17:43:38.652021 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:38.651986 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" event={"ID":"10d50cd8-583f-4b31-9667-4079d4a71a72","Type":"ContainerStarted","Data":"9a0441839feb71cd9bcc9c88e8882460574b9f9310ef8eef975291c4f8a6740e"} Jul 10 17:43:39.657632 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:39.657594 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6p5c" event={"ID":"946fb184-bf08-4f31-8ae3-37e1d2891cba","Type":"ContainerStarted","Data":"7e47405ad28c524f49936218b231cd503cbe68c3030f67d44844686651a4fb9b"} Jul 10 17:43:39.658062 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:39.657652 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6p5c" event={"ID":"946fb184-bf08-4f31-8ae3-37e1d2891cba","Type":"ContainerStarted","Data":"e2fbb806dd66bbb92023e3b1cc5a06af6b22cf985ec0d269072f1c54012023d6"} Jul 10 17:43:39.676984 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:39.676915 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h6p5c" podStartSLOduration=2.509749031 podStartE2EDuration="3.676895139s" totalImagesPullingTime="1.087975374s" totalInitContainerRuntime="79.170734ms" isStatefulPod=true podCreationTimestamp="2026-07-10 17:43:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:39.674946765 +0000 UTC m=+297.502595534" watchObservedRunningTime="2026-07-10 17:43:39.676895139 +0000 UTC m=+297.504543903" Jul 10 17:43:40.487406 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.486925 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5968457d4f-hz4qg"] Jul 10 17:43:40.493128 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.492131 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.495186 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.495135 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-f6twq\"" Jul 10 17:43:40.498443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.496696 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-78tajag60uovt\"" Jul 10 17:43:40.498443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.496952 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 10 17:43:40.498443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.497220 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 10 17:43:40.498443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.497238 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 10 17:43:40.498443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.497573 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 17:43:40.505886 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.505862 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5968457d4f-hz4qg"] Jul 10 17:43:40.518298 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.518271 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-secret-metrics-server-client-certs\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.518405 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.518323 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-secret-metrics-server-tls\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.518405 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.518354 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb488eba-86ea-4d04-a901-597cd1edce8f-metrics-server-audit-profiles\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.518528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.518415 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb488eba-86ea-4d04-a901-597cd1edce8f-audit-log\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.518528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.518487 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb488eba-86ea-4d04-a901-597cd1edce8f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.518528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.518512 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb7b2\" (UniqueName: \"kubernetes.io/projected/bb488eba-86ea-4d04-a901-597cd1edce8f-kube-api-access-mb7b2\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.518681 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.518571 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-client-ca-bundle\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.620005 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.619955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-secret-metrics-server-client-certs\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.620005 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.620007 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-secret-metrics-server-tls\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.620259 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.620043 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb488eba-86ea-4d04-a901-597cd1edce8f-metrics-server-audit-profiles\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.620321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.620248 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb488eba-86ea-4d04-a901-597cd1edce8f-audit-log\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.620381 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.620327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb488eba-86ea-4d04-a901-597cd1edce8f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.620381 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.620355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb7b2\" (UniqueName: \"kubernetes.io/projected/bb488eba-86ea-4d04-a901-597cd1edce8f-kube-api-access-mb7b2\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.620475 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.620393 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-client-ca-bundle\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.620698 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.620620 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/bb488eba-86ea-4d04-a901-597cd1edce8f-audit-log\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.621135 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.621112 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/bb488eba-86ea-4d04-a901-597cd1edce8f-metrics-server-audit-profiles\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.621403 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.621381 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb488eba-86ea-4d04-a901-597cd1edce8f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.622990 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.622949 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-secret-metrics-server-client-certs\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.623193 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.623172 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-client-ca-bundle\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.623300 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.623280 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/bb488eba-86ea-4d04-a901-597cd1edce8f-secret-metrics-server-tls\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.633769 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.633746 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb7b2\" (UniqueName: \"kubernetes.io/projected/bb488eba-86ea-4d04-a901-597cd1edce8f-kube-api-access-mb7b2\") pod \"metrics-server-5968457d4f-hz4qg\" (UID: \"bb488eba-86ea-4d04-a901-597cd1edce8f\") " pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:40.808741 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:40.808672 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:43:41.299085 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.299057 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7698767496-lf5z5"] Jul 10 17:43:41.302756 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.302728 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.305436 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.305409 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dprsp\"" Jul 10 17:43:41.305580 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.305488 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 17:43:41.305580 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.305538 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 17:43:41.305580 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.305488 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 17:43:41.305910 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.305870 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 17:43:41.306008 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.305927 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 17:43:41.310150 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.310125 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 17:43:41.316042 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.316019 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7698767496-lf5z5"] Jul 10 17:43:41.328163 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.328134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cgdw\" (UniqueName: \"kubernetes.io/projected/7dc2c7b4-ec4e-461f-9008-a574c38f458a-kube-api-access-7cgdw\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.328290 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.328186 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-federate-client-tls\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.328290 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.328222 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-serving-certs-ca-bundle\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.328290 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.328257 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.328430 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.328306 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.328430 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.328359 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-secret-telemeter-client\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.328430 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.328393 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-telemeter-client-tls\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.328572 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.328447 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-metrics-client-ca\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.428944 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.428906 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-telemeter-client-tls\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.429149 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.428980 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-metrics-client-ca\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.429149 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.429140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cgdw\" (UniqueName: \"kubernetes.io/projected/7dc2c7b4-ec4e-461f-9008-a574c38f458a-kube-api-access-7cgdw\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.429269 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.429189 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-federate-client-tls\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.429394 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.429363 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-serving-certs-ca-bundle\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.429459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.429426 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.429459 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.429452 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.429552 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.429494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-secret-telemeter-client\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.429873 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.429815 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-metrics-client-ca\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.430025 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.430008 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-serving-certs-ca-bundle\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.430378 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.430326 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dc2c7b4-ec4e-461f-9008-a574c38f458a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.432416 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.432397 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-federate-client-tls\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.432788 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.432726 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-secret-telemeter-client\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.433086 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.433064 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.433245 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.433230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7dc2c7b4-ec4e-461f-9008-a574c38f458a-telemeter-client-tls\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.437378 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.437357 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cgdw\" (UniqueName: \"kubernetes.io/projected/7dc2c7b4-ec4e-461f-9008-a574c38f458a-kube-api-access-7cgdw\") pod \"telemeter-client-7698767496-lf5z5\" (UID: \"7dc2c7b4-ec4e-461f-9008-a574c38f458a\") " pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:41.614540 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:41.614506 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" Jul 10 17:43:42.623644 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:42.623603 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7zqts" Jul 10 17:43:42.938582 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:42.938500 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68dd868d65-kkzvz"] Jul 10 17:43:42.967394 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:42.967361 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-559f68976c-9c9d4"] Jul 10 17:43:42.970520 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:42.970499 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:42.978531 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:42.978508 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 10 17:43:42.979201 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:42.979177 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-559f68976c-9c9d4"] Jul 10 17:43:43.044745 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.044707 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-trusted-ca-bundle\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.044910 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.044772 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-oauth-serving-cert\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.044910 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.044822 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j9xz\" (UniqueName: \"kubernetes.io/projected/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-kube-api-access-4j9xz\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.044910 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.044867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-oauth-config\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.044910 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.044887 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-config\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.045139 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.045000 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-serving-cert\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.045139 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.045032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-service-ca\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.146151 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.146111 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-config\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.146344 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.146184 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-serving-cert\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.146344 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.146215 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-service-ca\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.146344 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.146249 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-trusted-ca-bundle\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.146344 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.146299 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-oauth-serving-cert\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.146344 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.146339 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j9xz\" (UniqueName: \"kubernetes.io/projected/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-kube-api-access-4j9xz\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.146675 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.146395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-oauth-config\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.146937 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.146908 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-config\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.147168 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.147088 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-service-ca\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.147796 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.147771 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-trusted-ca-bundle\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.147917 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.147897 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-oauth-serving-cert\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.149152 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.149128 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-serving-cert\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.149253 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.149229 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-oauth-config\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.154208 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.154188 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j9xz\" (UniqueName: \"kubernetes.io/projected/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-kube-api-access-4j9xz\") pod \"console-559f68976c-9c9d4\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:43.281477 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:43.281404 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:46.248689 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.248152 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:43:46.250106 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.249437 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:43:46.254879 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.254855 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 17:43:46.356709 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.356488 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5968457d4f-hz4qg"] Jul 10 17:43:46.358019 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:46.357957 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb488eba_86ea_4d04_a901_597cd1edce8f.slice/crio-771ca418f90964e00ec27c5d91a1a5915ae0b6adb9e9114c22d9de4dbc382fe8 WatchSource:0}: Error finding container 771ca418f90964e00ec27c5d91a1a5915ae0b6adb9e9114c22d9de4dbc382fe8: Status 404 returned error can't find the container with id 771ca418f90964e00ec27c5d91a1a5915ae0b6adb9e9114c22d9de4dbc382fe8 Jul 10 17:43:46.361713 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.361695 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 17:43:46.596931 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.596097 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-559f68976c-9c9d4"] Jul 10 17:43:46.598527 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.598502 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7698767496-lf5z5"] Jul 10 17:43:46.600404 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:46.600203 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aee4e6b_f4c2_4383_b53f_b4161a02fbac.slice/crio-6b01a69dd5b13aac9653d17038a31e013d327e6b3f37b76a9684976c96120860 WatchSource:0}: Error finding container 6b01a69dd5b13aac9653d17038a31e013d327e6b3f37b76a9684976c96120860: Status 404 returned error can't find the container with id 6b01a69dd5b13aac9653d17038a31e013d327e6b3f37b76a9684976c96120860 Jul 10 17:43:46.692184 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.692151 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" event={"ID":"bb488eba-86ea-4d04-a901-597cd1edce8f","Type":"ContainerStarted","Data":"771ca418f90964e00ec27c5d91a1a5915ae0b6adb9e9114c22d9de4dbc382fe8"} Jul 10 17:43:46.693859 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.693827 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" event={"ID":"7dc2c7b4-ec4e-461f-9008-a574c38f458a","Type":"ContainerStarted","Data":"dafd11dfb484be51a9daa0afc6c3f9b9876b820c174da61f215a4df955dfb3db"} Jul 10 17:43:46.696302 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.696270 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559f68976c-9c9d4" event={"ID":"0aee4e6b-f4c2-4383-b53f-b4161a02fbac","Type":"ContainerStarted","Data":"09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce"} Jul 10 17:43:46.696393 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.696317 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559f68976c-9c9d4" event={"ID":"0aee4e6b-f4c2-4383-b53f-b4161a02fbac","Type":"ContainerStarted","Data":"6b01a69dd5b13aac9653d17038a31e013d327e6b3f37b76a9684976c96120860"} Jul 10 17:43:46.698914 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.698727 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-g86hh" event={"ID":"8f558d18-fa8d-4b98-8610-2427c33f7a4a","Type":"ContainerStarted","Data":"b87b30f8a38dd1c2b957014769e53a73e95385cb78ac50f17d80cb8e7c3bcd47"} Jul 10 17:43:46.699669 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.699375 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-g86hh" Jul 10 17:43:46.705559 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.705530 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-g86hh" Jul 10 17:43:46.714843 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.714790 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-559f68976c-9c9d4" podStartSLOduration=4.714771924 podStartE2EDuration="4.714771924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:46.713078084 +0000 UTC m=+304.540726866" watchObservedRunningTime="2026-07-10 17:43:46.714771924 +0000 UTC m=+304.542420693" Jul 10 17:43:46.729777 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:46.729726 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-g86hh" podStartSLOduration=1.52076573 podStartE2EDuration="19.729710902s" totalImagesPullingTime="18.208945172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:46.728598141 +0000 UTC m=+304.556246913" watchObservedRunningTime="2026-07-10 17:43:46.729710902 +0000 UTC m=+304.557359671" Jul 10 17:43:47.751017 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:47.750933 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:43:48.180245 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:48.180180 2568 patch_prober.go:28] interesting pod/image-registry-88d54bbc5-qgdnc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 17:43:48.180544 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:48.180248 2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" podUID="848cca2f-0571-43b3-bef7-0c67b5b013f9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 17:43:49.439935 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:49.438930 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-88d54bbc5-qgdnc"] Jul 10 17:43:49.446575 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:49.446545 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:43:51.733359 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:51.733319 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" event={"ID":"10d50cd8-583f-4b31-9667-4079d4a71a72","Type":"ContainerStarted","Data":"90f7e394d5687eb3773a72a0e71a9be08c953b72cf5a49220516e8439c1d27d2"} Jul 10 17:43:51.733815 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:51.733402 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" event={"ID":"10d50cd8-583f-4b31-9667-4079d4a71a72","Type":"ContainerStarted","Data":"e5d1ab505529ac174cf0ae4aa912ebff7de81d975832429e1f474f834affe755"} Jul 10 17:43:51.733815 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:51.733447 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" event={"ID":"10d50cd8-583f-4b31-9667-4079d4a71a72","Type":"ContainerStarted","Data":"29d1fe7d5ed829b98dec6cc2742b83faad76603c96b0634487d18a0539b755db"} Jul 10 17:43:51.735151 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:51.735114 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" event={"ID":"7dc2c7b4-ec4e-461f-9008-a574c38f458a","Type":"ContainerStarted","Data":"26ac857d96d97ed5008243fbf9d39477bf5377ff8e90f747257a2bf9f5a1aa6a"} Jul 10 17:43:51.737106 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:51.737023 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" event={"ID":"bb488eba-86ea-4d04-a901-597cd1edce8f","Type":"ContainerStarted","Data":"87c97232790cbc9901ef4338af62f693ac55230977e7a89ef3cc797d4645f669"} Jul 10 17:43:51.755815 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:51.755766 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" podStartSLOduration=7.405030752 podStartE2EDuration="11.755750398s" totalImagesPullingTime="4.350719646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:51.75451263 +0000 UTC m=+309.582161402" watchObservedRunningTime="2026-07-10 17:43:51.755750398 +0000 UTC m=+309.583399165" Jul 10 17:43:53.282089 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.282063 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:53.282455 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.282154 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:53.287889 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.287867 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:53.749314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.749229 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" event={"ID":"10d50cd8-583f-4b31-9667-4079d4a71a72","Type":"ContainerStarted","Data":"705828572db994e9e4cbae48a11f8b6506eabfa7db4dd81c8874a353a0ea41c1"} Jul 10 17:43:53.749487 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.749315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" event={"ID":"10d50cd8-583f-4b31-9667-4079d4a71a72","Type":"ContainerStarted","Data":"8a773d3f5bbcc6c2da56db63e9c87d7a590accd298b770fba892c61a5aab5197"} Jul 10 17:43:53.749487 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.749358 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" event={"ID":"10d50cd8-583f-4b31-9667-4079d4a71a72","Type":"ContainerStarted","Data":"e16bd1772f3cca22ead274a5e12c3f5cf0a22e5d59307b91eccd3485b51c704d"} Jul 10 17:43:53.749487 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.749409 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:43:53.751724 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.751697 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" event={"ID":"7dc2c7b4-ec4e-461f-9008-a574c38f458a","Type":"ContainerStarted","Data":"70cf0b0eba1711a239241a0a2dc8999e5328b7f027d34ac573b0338366dd5f45"} Jul 10 17:43:53.751845 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.751759 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" event={"ID":"7dc2c7b4-ec4e-461f-9008-a574c38f458a","Type":"ContainerStarted","Data":"215d4a52910e99839fe7bc0f821c8127eb53639e06f0a6fcda17c78234177caa"} Jul 10 17:43:53.756184 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.756161 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:43:53.770334 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.770295 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" podStartSLOduration=1.541093872 podStartE2EDuration="15.770275421s" totalImagesPullingTime="14.229181549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:53.769896984 +0000 UTC m=+311.597545751" watchObservedRunningTime="2026-07-10 17:43:53.770275421 +0000 UTC m=+311.597924190" Jul 10 17:43:53.790277 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:53.790235 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7698767496-lf5z5" podStartSLOduration=6.791069927 podStartE2EDuration="12.790221026s" totalImagesPullingTime="5.999151099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:53.78851743 +0000 UTC m=+311.616166199" watchObservedRunningTime="2026-07-10 17:43:53.790221026 +0000 UTC m=+311.617869793" Jul 10 17:43:54.605327 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.605279 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c95d65978-45d7s"] Jul 10 17:43:54.641746 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.641709 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c95d65978-45d7s"] Jul 10 17:43:54.641912 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.641842 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.776181 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.776142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-oauth-serving-cert\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.776181 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.776185 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drq67\" (UniqueName: \"kubernetes.io/projected/e6507faa-3ab0-4714-b5b8-4634d7092bd8-kube-api-access-drq67\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.776435 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.776237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-config\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.776435 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.776328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-service-ca\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.776435 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.776346 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-trusted-ca-bundle\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.776435 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.776373 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-serving-cert\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.776435 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.776406 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-oauth-config\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.878166 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.878079 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-service-ca\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.878166 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.878126 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-trusted-ca-bundle\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.878166 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.878164 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-serving-cert\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.878447 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.878239 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-oauth-config\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.878447 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.878256 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-oauth-serving-cert\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.878447 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.878271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drq67\" (UniqueName: \"kubernetes.io/projected/e6507faa-3ab0-4714-b5b8-4634d7092bd8-kube-api-access-drq67\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.878447 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.878402 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-config\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.879151 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.879124 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-config\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.879610 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.879581 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-service-ca\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.879904 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.879882 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-oauth-serving-cert\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.880336 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.880311 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-trusted-ca-bundle\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.882919 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.882896 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-serving-cert\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.883039 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.882933 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-oauth-config\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.886868 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.886843 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drq67\" (UniqueName: \"kubernetes.io/projected/e6507faa-3ab0-4714-b5b8-4634d7092bd8-kube-api-access-drq67\") pod \"console-7c95d65978-45d7s\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:54.954593 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:54.954559 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:43:55.133280 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:55.133216 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c95d65978-45d7s"] Jul 10 17:43:55.135472 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:43:55.135437 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6507faa_3ab0_4714_b5b8_4634d7092bd8.slice/crio-a0a4639ff043b0a182157fa4d203fc9de832785c55c7455bfb5dc3c41c1b8dda WatchSource:0}: Error finding container a0a4639ff043b0a182157fa4d203fc9de832785c55c7455bfb5dc3c41c1b8dda: Status 404 returned error can't find the container with id a0a4639ff043b0a182157fa4d203fc9de832785c55c7455bfb5dc3c41c1b8dda Jul 10 17:43:55.762360 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:55.762326 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c95d65978-45d7s" event={"ID":"e6507faa-3ab0-4714-b5b8-4634d7092bd8","Type":"ContainerStarted","Data":"8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e"} Jul 10 17:43:55.762707 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:55.762400 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c95d65978-45d7s" event={"ID":"e6507faa-3ab0-4714-b5b8-4634d7092bd8","Type":"ContainerStarted","Data":"a0a4639ff043b0a182157fa4d203fc9de832785c55c7455bfb5dc3c41c1b8dda"} Jul 10 17:43:55.778316 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:55.778258 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c95d65978-45d7s" podStartSLOduration=1.778240182 podStartE2EDuration="1.778240182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:43:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:43:55.778022071 +0000 UTC m=+313.605670840" watchObservedRunningTime="2026-07-10 17:43:55.778240182 +0000 UTC m=+313.605888951" Jul 10 17:43:59.763760 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:43:59.763734 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-655b67cf94-vb9rq" Jul 10 17:44:00.809184 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:00.809152 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:44:00.809603 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:00.809200 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:44:04.955718 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:04.955688 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:44:04.956208 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:04.955800 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:44:04.960599 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:04.960574 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:44:05.813887 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:05.813857 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:44:05.856673 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:05.856647 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-559f68976c-9c9d4"] Jul 10 17:44:07.968681 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:07.968644 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68dd868d65-kkzvz" podUID="61eb2013-8a8d-4e81-adff-5ff31f6a766e" containerName="console" containerID="cri-o://733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268" gracePeriod=15 Jul 10 17:44:08.243470 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.243411 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:44:08.397996 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.397951 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-config\") pod \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " Jul 10 17:44:08.398162 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398014 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-oauth-config\") pod \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " Jul 10 17:44:08.398253 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398171 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vxhp\" (UniqueName: \"kubernetes.io/projected/61eb2013-8a8d-4e81-adff-5ff31f6a766e-kube-api-access-9vxhp\") pod \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " Jul 10 17:44:08.398253 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398221 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-oauth-serving-cert\") pod \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " Jul 10 17:44:08.398340 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398274 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-service-ca\") pod \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " Jul 10 17:44:08.398340 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398302 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-serving-cert\") pod \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\" (UID: \"61eb2013-8a8d-4e81-adff-5ff31f6a766e\") " Jul 10 17:44:08.398442 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398354 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-config" (OuterVolumeSpecName: "console-config") pod "61eb2013-8a8d-4e81-adff-5ff31f6a766e" (UID: "61eb2013-8a8d-4e81-adff-5ff31f6a766e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:08.398672 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398577 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:08.398749 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398679 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "61eb2013-8a8d-4e81-adff-5ff31f6a766e" (UID: "61eb2013-8a8d-4e81-adff-5ff31f6a766e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:08.398787 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.398763 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-service-ca" (OuterVolumeSpecName: "service-ca") pod "61eb2013-8a8d-4e81-adff-5ff31f6a766e" (UID: "61eb2013-8a8d-4e81-adff-5ff31f6a766e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:08.400739 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.400717 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "61eb2013-8a8d-4e81-adff-5ff31f6a766e" (UID: "61eb2013-8a8d-4e81-adff-5ff31f6a766e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:44:08.400892 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.400872 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "61eb2013-8a8d-4e81-adff-5ff31f6a766e" (UID: "61eb2013-8a8d-4e81-adff-5ff31f6a766e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:44:08.400938 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.400882 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61eb2013-8a8d-4e81-adff-5ff31f6a766e-kube-api-access-9vxhp" (OuterVolumeSpecName: "kube-api-access-9vxhp") pod "61eb2013-8a8d-4e81-adff-5ff31f6a766e" (UID: "61eb2013-8a8d-4e81-adff-5ff31f6a766e"). InnerVolumeSpecName "kube-api-access-9vxhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:44:08.499552 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.499490 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-oauth-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:08.499552 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.499516 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61eb2013-8a8d-4e81-adff-5ff31f6a766e-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:08.499552 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.499529 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:08.499552 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.499542 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61eb2013-8a8d-4e81-adff-5ff31f6a766e-console-oauth-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:08.499759 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.499556 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vxhp\" (UniqueName: \"kubernetes.io/projected/61eb2013-8a8d-4e81-adff-5ff31f6a766e-kube-api-access-9vxhp\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:08.811636 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.811603 2568 generic.go:356] "Generic (PLEG): container finished" podID="61eb2013-8a8d-4e81-adff-5ff31f6a766e" containerID="733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268" exitCode=0 Jul 10 17:44:08.811806 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.811659 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68dd868d65-kkzvz" event={"ID":"61eb2013-8a8d-4e81-adff-5ff31f6a766e","Type":"ContainerDied","Data":"733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268"} Jul 10 17:44:08.811806 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.811702 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68dd868d65-kkzvz" event={"ID":"61eb2013-8a8d-4e81-adff-5ff31f6a766e","Type":"ContainerDied","Data":"f2109f1ab26ac330ccf3afc74836277653a5b7751526f45bbb588136aff69988"} Jul 10 17:44:08.811806 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.811647 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68dd868d65-kkzvz" Jul 10 17:44:08.811806 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.811745 2568 scope.go:117] "RemoveContainer" containerID="733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268" Jul 10 17:44:08.827269 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.827242 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68dd868d65-kkzvz"] Jul 10 17:44:08.829216 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.829200 2568 scope.go:117] "RemoveContainer" containerID="733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268" Jul 10 17:44:08.829510 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:44:08.829491 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268\": container with ID starting with 733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268 not found: ID does not exist" containerID="733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268" Jul 10 17:44:08.829558 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.829518 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268"} err="failed to get container status \"733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268\": rpc error: code = NotFound desc = could not find container \"733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268\": container with ID starting with 733476bbc282485143b83c2d0fe22aa5881c01e18581a92f37b235cb492d6268 not found: ID does not exist" Jul 10 17:44:08.833257 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:08.833238 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68dd868d65-kkzvz"] Jul 10 17:44:10.755564 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:10.755531 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61eb2013-8a8d-4e81-adff-5ff31f6a766e" path="/var/lib/kubelet/pods/61eb2013-8a8d-4e81-adff-5ff31f6a766e/volumes" Jul 10 17:44:13.835768 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:13.835737 2568 generic.go:356] "Generic (PLEG): container finished" podID="ea8f305d-b4f9-4c84-98b7-ce6ad74f3006" containerID="ea8bb3da197f88b057ca6c18559389bd3407b15ba03880940ae5ba76fb9b6dc3" exitCode=0 Jul 10 17:44:13.836205 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:13.835791 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" event={"ID":"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006","Type":"ContainerDied","Data":"ea8bb3da197f88b057ca6c18559389bd3407b15ba03880940ae5ba76fb9b6dc3"} Jul 10 17:44:13.836205 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:13.836104 2568 scope.go:117] "RemoveContainer" containerID="ea8bb3da197f88b057ca6c18559389bd3407b15ba03880940ae5ba76fb9b6dc3" Jul 10 17:44:14.467238 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.467180 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" podUID="848cca2f-0571-43b3-bef7-0c67b5b013f9" containerName="registry" containerID="cri-o://6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348" gracePeriod=30 Jul 10 17:44:14.774023 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.773997 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:44:14.841398 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.841364 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-jxbvt" event={"ID":"ea8f305d-b4f9-4c84-98b7-ce6ad74f3006","Type":"ContainerStarted","Data":"e248cce3031d8eb9058a900d7b4e4407c6e860aca36e17a61878d814501ca33e"} Jul 10 17:44:14.842565 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.842540 2568 generic.go:356] "Generic (PLEG): container finished" podID="848cca2f-0571-43b3-bef7-0c67b5b013f9" containerID="6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348" exitCode=0 Jul 10 17:44:14.842694 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.842574 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" Jul 10 17:44:14.842694 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.842587 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" event={"ID":"848cca2f-0571-43b3-bef7-0c67b5b013f9","Type":"ContainerDied","Data":"6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348"} Jul 10 17:44:14.842694 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.842631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-88d54bbc5-qgdnc" event={"ID":"848cca2f-0571-43b3-bef7-0c67b5b013f9","Type":"ContainerDied","Data":"501fc13552e0c996eccae2f332fb029528c07c9db4b3ba4e0bc061c3c0b6b4f4"} Jul 10 17:44:14.842694 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.842658 2568 scope.go:117] "RemoveContainer" containerID="6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348" Jul 10 17:44:14.851161 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.851146 2568 scope.go:117] "RemoveContainer" containerID="6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348" Jul 10 17:44:14.851382 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:44:14.851366 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348\": container with ID starting with 6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348 not found: ID does not exist" containerID="6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348" Jul 10 17:44:14.851421 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.851393 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348"} err="failed to get container status \"6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348\": rpc error: code = NotFound desc = could not find container \"6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348\": container with ID starting with 6f04ca5357b1d4d9851f7021c5b9c77161a323517ac4e6893a54350d52cfa348 not found: ID does not exist" Jul 10 17:44:14.853699 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.853685 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-certificates\") pod \"848cca2f-0571-43b3-bef7-0c67b5b013f9\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " Jul 10 17:44:14.853745 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.853723 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-trusted-ca\") pod \"848cca2f-0571-43b3-bef7-0c67b5b013f9\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " Jul 10 17:44:14.853788 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.853748 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/848cca2f-0571-43b3-bef7-0c67b5b013f9-ca-trust-extracted\") pod \"848cca2f-0571-43b3-bef7-0c67b5b013f9\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " Jul 10 17:44:14.853788 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.853778 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-image-registry-private-configuration\") pod \"848cca2f-0571-43b3-bef7-0c67b5b013f9\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " Jul 10 17:44:14.853880 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.853795 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-installation-pull-secrets\") pod \"848cca2f-0571-43b3-bef7-0c67b5b013f9\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " Jul 10 17:44:14.853880 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.853811 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-bound-sa-token\") pod \"848cca2f-0571-43b3-bef7-0c67b5b013f9\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " Jul 10 17:44:14.853880 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.853869 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cpph\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-kube-api-access-5cpph\") pod \"848cca2f-0571-43b3-bef7-0c67b5b013f9\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " Jul 10 17:44:14.854023 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.853911 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") pod \"848cca2f-0571-43b3-bef7-0c67b5b013f9\" (UID: \"848cca2f-0571-43b3-bef7-0c67b5b013f9\") " Jul 10 17:44:14.854324 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.854196 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "848cca2f-0571-43b3-bef7-0c67b5b013f9" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:14.854324 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.854208 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "848cca2f-0571-43b3-bef7-0c67b5b013f9" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:14.857299 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.857176 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-kube-api-access-5cpph" (OuterVolumeSpecName: "kube-api-access-5cpph") pod "848cca2f-0571-43b3-bef7-0c67b5b013f9" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9"). InnerVolumeSpecName "kube-api-access-5cpph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:44:14.857493 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.857461 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "848cca2f-0571-43b3-bef7-0c67b5b013f9" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:44:14.857652 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.857618 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "848cca2f-0571-43b3-bef7-0c67b5b013f9" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:44:14.857754 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.857652 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "848cca2f-0571-43b3-bef7-0c67b5b013f9" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:44:14.857754 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.857656 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "848cca2f-0571-43b3-bef7-0c67b5b013f9" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:44:14.864695 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.864667 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/848cca2f-0571-43b3-bef7-0c67b5b013f9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "848cca2f-0571-43b3-bef7-0c67b5b013f9" (UID: "848cca2f-0571-43b3-bef7-0c67b5b013f9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:44:14.954820 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.954794 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:14.954820 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.954818 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-registry-certificates\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:14.954820 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.954828 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/848cca2f-0571-43b3-bef7-0c67b5b013f9-trusted-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:14.955048 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.954836 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/848cca2f-0571-43b3-bef7-0c67b5b013f9-ca-trust-extracted\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:14.955048 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.954845 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-image-registry-private-configuration\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:14.955048 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.954854 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/848cca2f-0571-43b3-bef7-0c67b5b013f9-installation-pull-secrets\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:14.955048 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.954862 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-bound-sa-token\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:14.955048 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:14.954870 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cpph\" (UniqueName: \"kubernetes.io/projected/848cca2f-0571-43b3-bef7-0c67b5b013f9-kube-api-access-5cpph\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:15.163238 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:15.163209 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-88d54bbc5-qgdnc"] Jul 10 17:44:15.166399 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:15.166372 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-88d54bbc5-qgdnc"] Jul 10 17:44:16.755058 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:16.755027 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="848cca2f-0571-43b3-bef7-0c67b5b013f9" path="/var/lib/kubelet/pods/848cca2f-0571-43b3-bef7-0c67b5b013f9/volumes" Jul 10 17:44:20.814790 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:20.814760 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:44:20.818761 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:20.818742 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5968457d4f-hz4qg" Jul 10 17:44:30.884877 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:30.884821 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-559f68976c-9c9d4" podUID="0aee4e6b-f4c2-4383-b53f-b4161a02fbac" containerName="console" containerID="cri-o://09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce" gracePeriod=15 Jul 10 17:44:31.159702 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.159637 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:44:31.291737 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.291706 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-oauth-config\") pod \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " Jul 10 17:44:31.291917 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.291748 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-oauth-serving-cert\") pod \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " Jul 10 17:44:31.291917 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.291783 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-config\") pod \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " Jul 10 17:44:31.291917 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.291807 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-serving-cert\") pod \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " Jul 10 17:44:31.291917 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.291833 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-trusted-ca-bundle\") pod \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " Jul 10 17:44:31.291917 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.291887 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-service-ca\") pod \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " Jul 10 17:44:31.291917 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.291918 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j9xz\" (UniqueName: \"kubernetes.io/projected/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-kube-api-access-4j9xz\") pod \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\" (UID: \"0aee4e6b-f4c2-4383-b53f-b4161a02fbac\") " Jul 10 17:44:31.292288 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.292254 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0aee4e6b-f4c2-4383-b53f-b4161a02fbac" (UID: "0aee4e6b-f4c2-4383-b53f-b4161a02fbac"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:31.292349 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.292280 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-config" (OuterVolumeSpecName: "console-config") pod "0aee4e6b-f4c2-4383-b53f-b4161a02fbac" (UID: "0aee4e6b-f4c2-4383-b53f-b4161a02fbac"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:31.292389 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.292363 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-service-ca" (OuterVolumeSpecName: "service-ca") pod "0aee4e6b-f4c2-4383-b53f-b4161a02fbac" (UID: "0aee4e6b-f4c2-4383-b53f-b4161a02fbac"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:31.292389 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.292372 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0aee4e6b-f4c2-4383-b53f-b4161a02fbac" (UID: "0aee4e6b-f4c2-4383-b53f-b4161a02fbac"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:44:31.294177 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.294150 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0aee4e6b-f4c2-4383-b53f-b4161a02fbac" (UID: "0aee4e6b-f4c2-4383-b53f-b4161a02fbac"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:44:31.294278 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.294188 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0aee4e6b-f4c2-4383-b53f-b4161a02fbac" (UID: "0aee4e6b-f4c2-4383-b53f-b4161a02fbac"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:44:31.294278 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.294252 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-kube-api-access-4j9xz" (OuterVolumeSpecName: "kube-api-access-4j9xz") pod "0aee4e6b-f4c2-4383-b53f-b4161a02fbac" (UID: "0aee4e6b-f4c2-4383-b53f-b4161a02fbac"). InnerVolumeSpecName "kube-api-access-4j9xz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:44:31.392815 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.392790 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-trusted-ca-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:31.392815 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.392812 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:31.392997 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.392823 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j9xz\" (UniqueName: \"kubernetes.io/projected/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-kube-api-access-4j9xz\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:31.392997 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.392832 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-oauth-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:31.392997 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.392841 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-oauth-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:31.392997 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.392851 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:31.392997 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.392859 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0aee4e6b-f4c2-4383-b53f-b4161a02fbac-console-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:44:31.909067 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.909037 2568 generic.go:356] "Generic (PLEG): container finished" podID="0aee4e6b-f4c2-4383-b53f-b4161a02fbac" containerID="09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce" exitCode=0 Jul 10 17:44:31.909601 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.909087 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-559f68976c-9c9d4" Jul 10 17:44:31.909601 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.909126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559f68976c-9c9d4" event={"ID":"0aee4e6b-f4c2-4383-b53f-b4161a02fbac","Type":"ContainerDied","Data":"09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce"} Jul 10 17:44:31.909601 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.909194 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-559f68976c-9c9d4" event={"ID":"0aee4e6b-f4c2-4383-b53f-b4161a02fbac","Type":"ContainerDied","Data":"6b01a69dd5b13aac9653d17038a31e013d327e6b3f37b76a9684976c96120860"} Jul 10 17:44:31.909601 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.909227 2568 scope.go:117] "RemoveContainer" containerID="09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce" Jul 10 17:44:31.918127 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.918108 2568 scope.go:117] "RemoveContainer" containerID="09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce" Jul 10 17:44:31.918391 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:44:31.918374 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce\": container with ID starting with 09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce not found: ID does not exist" containerID="09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce" Jul 10 17:44:31.918432 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.918402 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce"} err="failed to get container status \"09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce\": rpc error: code = NotFound desc = could not find container \"09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce\": container with ID starting with 09726d2cbf892ed4a68edf1cd2871aee94e243ac4ddd93017beda71e93e145ce not found: ID does not exist" Jul 10 17:44:31.929173 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.929154 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-559f68976c-9c9d4"] Jul 10 17:44:31.933150 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:31.933117 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-559f68976c-9c9d4"] Jul 10 17:44:32.754313 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:32.754280 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0aee4e6b-f4c2-4383-b53f-b4161a02fbac" path="/var/lib/kubelet/pods/0aee4e6b-f4c2-4383-b53f-b4161a02fbac/volumes" Jul 10 17:44:56.611820 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:56.611781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:44:56.614241 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:56.614222 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d209f2ef-b7c8-4817-8950-548fbf79929d-metrics-certs\") pod \"network-metrics-daemon-l5nwq\" (UID: \"d209f2ef-b7c8-4817-8950-548fbf79929d\") " pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:44:56.756367 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:56.756330 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rxjk8\"" Jul 10 17:44:56.762352 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:56.762328 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l5nwq" Jul 10 17:44:56.895422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:56.895338 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l5nwq"] Jul 10 17:44:56.898076 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:44:56.898051 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd209f2ef_b7c8_4817_8950_548fbf79929d.slice/crio-7f8196481257e5ae45030ab19a4a8fe54ec89dff95648e8c0cc1c59dde0f3ef6 WatchSource:0}: Error finding container 7f8196481257e5ae45030ab19a4a8fe54ec89dff95648e8c0cc1c59dde0f3ef6: Status 404 returned error can't find the container with id 7f8196481257e5ae45030ab19a4a8fe54ec89dff95648e8c0cc1c59dde0f3ef6 Jul 10 17:44:57.002729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:57.002697 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l5nwq" event={"ID":"d209f2ef-b7c8-4817-8950-548fbf79929d","Type":"ContainerStarted","Data":"7f8196481257e5ae45030ab19a4a8fe54ec89dff95648e8c0cc1c59dde0f3ef6"} Jul 10 17:44:58.006997 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:58.006956 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l5nwq" event={"ID":"d209f2ef-b7c8-4817-8950-548fbf79929d","Type":"ContainerStarted","Data":"91e84e929c260a26cc6ea6c86b522e62cc6de2193f01d663a37206e9f838c760"} Jul 10 17:44:59.012629 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:59.012593 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l5nwq" event={"ID":"d209f2ef-b7c8-4817-8950-548fbf79929d","Type":"ContainerStarted","Data":"ff944af983cccdac2fe2a4e126054691671cf3830380ecd821dce93013fec56b"} Jul 10 17:44:59.026750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:44:59.026701 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-l5nwq" podStartSLOduration=375.051860359 podStartE2EDuration="6m16.026686762s" totalImagesPullingTime="974.826403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:44:59.026587931 +0000 UTC m=+376.854236699" watchObservedRunningTime="2026-07-10 17:44:59.026686762 +0000 UTC m=+376.854335530" Jul 10 17:45:30.800747 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:30.800715 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c95d65978-45d7s"] Jul 10 17:45:55.824095 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:55.824038 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c95d65978-45d7s" podUID="e6507faa-3ab0-4714-b5b8-4634d7092bd8" containerName="console" containerID="cri-o://8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e" gracePeriod=15 Jul 10 17:45:56.072064 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.072043 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:45:56.166429 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.166403 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-trusted-ca-bundle\") pod \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " Jul 10 17:45:56.166608 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.166455 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-serving-cert\") pod \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " Jul 10 17:45:56.166608 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.166511 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-oauth-config\") pod \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " Jul 10 17:45:56.166608 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.166563 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-oauth-serving-cert\") pod \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " Jul 10 17:45:56.166608 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.166585 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drq67\" (UniqueName: \"kubernetes.io/projected/e6507faa-3ab0-4714-b5b8-4634d7092bd8-kube-api-access-drq67\") pod \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " Jul 10 17:45:56.166608 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.166607 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-config\") pod \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " Jul 10 17:45:56.166846 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.166640 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-service-ca\") pod \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\" (UID: \"e6507faa-3ab0-4714-b5b8-4634d7092bd8\") " Jul 10 17:45:56.166846 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.166797 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e6507faa-3ab0-4714-b5b8-4634d7092bd8" (UID: "e6507faa-3ab0-4714-b5b8-4634d7092bd8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:45:56.167078 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.167045 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-trusted-ca-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:45:56.167078 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.167055 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e6507faa-3ab0-4714-b5b8-4634d7092bd8" (UID: "e6507faa-3ab0-4714-b5b8-4634d7092bd8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:45:56.167078 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.167068 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-config" (OuterVolumeSpecName: "console-config") pod "e6507faa-3ab0-4714-b5b8-4634d7092bd8" (UID: "e6507faa-3ab0-4714-b5b8-4634d7092bd8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:45:56.167475 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.167124 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-service-ca" (OuterVolumeSpecName: "service-ca") pod "e6507faa-3ab0-4714-b5b8-4634d7092bd8" (UID: "e6507faa-3ab0-4714-b5b8-4634d7092bd8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:45:56.168795 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.168768 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e6507faa-3ab0-4714-b5b8-4634d7092bd8" (UID: "e6507faa-3ab0-4714-b5b8-4634d7092bd8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:45:56.168891 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.168834 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6507faa-3ab0-4714-b5b8-4634d7092bd8-kube-api-access-drq67" (OuterVolumeSpecName: "kube-api-access-drq67") pod "e6507faa-3ab0-4714-b5b8-4634d7092bd8" (UID: "e6507faa-3ab0-4714-b5b8-4634d7092bd8"). InnerVolumeSpecName "kube-api-access-drq67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:45:56.168891 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.168857 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e6507faa-3ab0-4714-b5b8-4634d7092bd8" (UID: "e6507faa-3ab0-4714-b5b8-4634d7092bd8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:45:56.225438 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.225413 2568 generic.go:356] "Generic (PLEG): container finished" podID="e6507faa-3ab0-4714-b5b8-4634d7092bd8" containerID="8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e" exitCode=0 Jul 10 17:45:56.225535 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.225456 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c95d65978-45d7s" Jul 10 17:45:56.225535 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.225500 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c95d65978-45d7s" event={"ID":"e6507faa-3ab0-4714-b5b8-4634d7092bd8","Type":"ContainerDied","Data":"8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e"} Jul 10 17:45:56.225607 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.225548 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c95d65978-45d7s" event={"ID":"e6507faa-3ab0-4714-b5b8-4634d7092bd8","Type":"ContainerDied","Data":"a0a4639ff043b0a182157fa4d203fc9de832785c55c7455bfb5dc3c41c1b8dda"} Jul 10 17:45:56.225607 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.225569 2568 scope.go:117] "RemoveContainer" containerID="8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e" Jul 10 17:45:56.233935 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.233921 2568 scope.go:117] "RemoveContainer" containerID="8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e" Jul 10 17:45:56.234269 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:45:56.234251 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e\": container with ID starting with 8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e not found: ID does not exist" containerID="8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e" Jul 10 17:45:56.234316 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.234276 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e"} err="failed to get container status \"8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e\": rpc error: code = NotFound desc = could not find container \"8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e\": container with ID starting with 8feccb588010cba0c7bcf70ec751cf074d8855ffd980786b2a49414623e0f37e not found: ID does not exist" Jul 10 17:45:56.245013 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.244990 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c95d65978-45d7s"] Jul 10 17:45:56.246846 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.246826 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c95d65978-45d7s"] Jul 10 17:45:56.267872 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.267855 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-oauth-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:45:56.267940 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.267875 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drq67\" (UniqueName: \"kubernetes.io/projected/e6507faa-3ab0-4714-b5b8-4634d7092bd8-kube-api-access-drq67\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:45:56.267940 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.267884 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:45:56.267940 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.267893 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e6507faa-3ab0-4714-b5b8-4634d7092bd8-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:45:56.267940 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.267901 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-serving-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:45:56.267940 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.267909 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e6507faa-3ab0-4714-b5b8-4634d7092bd8-console-oauth-config\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:45:56.755174 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:45:56.755143 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6507faa-3ab0-4714-b5b8-4634d7092bd8" path="/var/lib/kubelet/pods/e6507faa-3ab0-4714-b5b8-4634d7092bd8/volumes" Jul 10 17:46:16.667437 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667405 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7"] Jul 10 17:46:16.667813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667706 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aee4e6b-f4c2-4383-b53f-b4161a02fbac" containerName="console" Jul 10 17:46:16.667813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667721 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aee4e6b-f4c2-4383-b53f-b4161a02fbac" containerName="console" Jul 10 17:46:16.667813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667741 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6507faa-3ab0-4714-b5b8-4634d7092bd8" containerName="console" Jul 10 17:46:16.667813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667746 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6507faa-3ab0-4714-b5b8-4634d7092bd8" containerName="console" Jul 10 17:46:16.667813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667757 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="848cca2f-0571-43b3-bef7-0c67b5b013f9" containerName="registry" Jul 10 17:46:16.667813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667763 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="848cca2f-0571-43b3-bef7-0c67b5b013f9" containerName="registry" Jul 10 17:46:16.667813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667769 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61eb2013-8a8d-4e81-adff-5ff31f6a766e" containerName="console" Jul 10 17:46:16.667813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667774 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="61eb2013-8a8d-4e81-adff-5ff31f6a766e" containerName="console" Jul 10 17:46:16.668064 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667829 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="61eb2013-8a8d-4e81-adff-5ff31f6a766e" containerName="console" Jul 10 17:46:16.668064 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667837 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="848cca2f-0571-43b3-bef7-0c67b5b013f9" containerName="registry" Jul 10 17:46:16.668064 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667845 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6507faa-3ab0-4714-b5b8-4634d7092bd8" containerName="console" Jul 10 17:46:16.668064 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.667851 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aee4e6b-f4c2-4383-b53f-b4161a02fbac" containerName="console" Jul 10 17:46:16.670935 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.670919 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.673576 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.673552 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 17:46:16.673705 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.673552 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 17:46:16.674389 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.674378 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cj2k7\"" Jul 10 17:46:16.682668 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.682647 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7"] Jul 10 17:46:16.724787 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.724758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.724907 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.724815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.724907 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.724856 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8rgx\" (UniqueName: \"kubernetes.io/projected/ed0cd39b-d0ca-4dfa-8794-e429691d9991-kube-api-access-n8rgx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.825511 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.825487 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.825646 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.825552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.825646 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.825592 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8rgx\" (UniqueName: \"kubernetes.io/projected/ed0cd39b-d0ca-4dfa-8794-e429691d9991-kube-api-access-n8rgx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.825899 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.825876 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.825940 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.825910 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.833525 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.833495 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8rgx\" (UniqueName: \"kubernetes.io/projected/ed0cd39b-d0ca-4dfa-8794-e429691d9991-kube-api-access-n8rgx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:16.979837 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:16.979735 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:17.115321 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:17.115287 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7"] Jul 10 17:46:17.118154 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:46:17.118125 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0cd39b_d0ca_4dfa_8794_e429691d9991.slice/crio-5a5b74f349b1245b3f6e8a6ad0f39738af54a9bc7e73967e59ab840bbbb8c106 WatchSource:0}: Error finding container 5a5b74f349b1245b3f6e8a6ad0f39738af54a9bc7e73967e59ab840bbbb8c106: Status 404 returned error can't find the container with id 5a5b74f349b1245b3f6e8a6ad0f39738af54a9bc7e73967e59ab840bbbb8c106 Jul 10 17:46:17.307074 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:17.306999 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" event={"ID":"ed0cd39b-d0ca-4dfa-8794-e429691d9991","Type":"ContainerStarted","Data":"5a5b74f349b1245b3f6e8a6ad0f39738af54a9bc7e73967e59ab840bbbb8c106"} Jul 10 17:46:25.341875 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:25.341843 2568 generic.go:356] "Generic (PLEG): container finished" podID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerID="7052d6a894916a8a4fb95ff7fb781a8ac13314c084ee1e07d227b37da606d591" exitCode=0 Jul 10 17:46:25.342273 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:25.341910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" event={"ID":"ed0cd39b-d0ca-4dfa-8794-e429691d9991","Type":"ContainerDied","Data":"7052d6a894916a8a4fb95ff7fb781a8ac13314c084ee1e07d227b37da606d591"} Jul 10 17:46:28.355352 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:28.355316 2568 generic.go:356] "Generic (PLEG): container finished" podID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerID="b378af99c8124743ccd985d907fd44639c0e1e37e134b70fc682849cc7ddff69" exitCode=0 Jul 10 17:46:28.355740 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:28.355386 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" event={"ID":"ed0cd39b-d0ca-4dfa-8794-e429691d9991","Type":"ContainerDied","Data":"b378af99c8124743ccd985d907fd44639c0e1e37e134b70fc682849cc7ddff69"} Jul 10 17:46:36.393395 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:36.393359 2568 generic.go:356] "Generic (PLEG): container finished" podID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerID="63335a94e05557958795057e3fdf301ad02565ac29b62822b52a27defd799499" exitCode=0 Jul 10 17:46:36.393787 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:36.393433 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" event={"ID":"ed0cd39b-d0ca-4dfa-8794-e429691d9991","Type":"ContainerDied","Data":"63335a94e05557958795057e3fdf301ad02565ac29b62822b52a27defd799499"} Jul 10 17:46:37.569705 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.569675 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:37.699034 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.698940 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-util\") pod \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " Jul 10 17:46:37.699186 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.699041 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-bundle\") pod \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " Jul 10 17:46:37.699186 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.699085 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8rgx\" (UniqueName: \"kubernetes.io/projected/ed0cd39b-d0ca-4dfa-8794-e429691d9991-kube-api-access-n8rgx\") pod \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\" (UID: \"ed0cd39b-d0ca-4dfa-8794-e429691d9991\") " Jul 10 17:46:37.699636 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.699612 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-bundle" (OuterVolumeSpecName: "bundle") pod "ed0cd39b-d0ca-4dfa-8794-e429691d9991" (UID: "ed0cd39b-d0ca-4dfa-8794-e429691d9991"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:46:37.701338 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.701310 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed0cd39b-d0ca-4dfa-8794-e429691d9991-kube-api-access-n8rgx" (OuterVolumeSpecName: "kube-api-access-n8rgx") pod "ed0cd39b-d0ca-4dfa-8794-e429691d9991" (UID: "ed0cd39b-d0ca-4dfa-8794-e429691d9991"). InnerVolumeSpecName "kube-api-access-n8rgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:46:37.705924 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.705899 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-util" (OuterVolumeSpecName: "util") pod "ed0cd39b-d0ca-4dfa-8794-e429691d9991" (UID: "ed0cd39b-d0ca-4dfa-8794-e429691d9991"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:46:37.799673 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.799644 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8rgx\" (UniqueName: \"kubernetes.io/projected/ed0cd39b-d0ca-4dfa-8794-e429691d9991-kube-api-access-n8rgx\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:46:37.799673 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.799668 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-util\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:46:37.799673 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:37.799678 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ed0cd39b-d0ca-4dfa-8794-e429691d9991-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:46:38.403457 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:38.403433 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" Jul 10 17:46:38.403680 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:38.403459 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540z44w7" event={"ID":"ed0cd39b-d0ca-4dfa-8794-e429691d9991","Type":"ContainerDied","Data":"5a5b74f349b1245b3f6e8a6ad0f39738af54a9bc7e73967e59ab840bbbb8c106"} Jul 10 17:46:38.403680 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:38.403500 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a5b74f349b1245b3f6e8a6ad0f39738af54a9bc7e73967e59ab840bbbb8c106" Jul 10 17:46:44.787192 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.787161 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79"] Jul 10 17:46:44.787541 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.787456 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerName="util" Jul 10 17:46:44.787541 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.787467 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerName="util" Jul 10 17:46:44.787541 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.787475 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerName="pull" Jul 10 17:46:44.787541 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.787481 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerName="pull" Jul 10 17:46:44.787541 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.787504 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerName="extract" Jul 10 17:46:44.787541 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.787513 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerName="extract" Jul 10 17:46:44.787719 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.787569 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed0cd39b-d0ca-4dfa-8794-e429691d9991" containerName="extract" Jul 10 17:46:44.791340 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.791325 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" Jul 10 17:46:44.794793 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.794764 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 10 17:46:44.795669 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.795636 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 10 17:46:44.800333 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.799239 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-585kc\"" Jul 10 17:46:44.812705 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.812684 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79"] Jul 10 17:46:44.852298 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.852270 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8lwv\" (UniqueName: \"kubernetes.io/projected/3035f933-a8bc-4e54-9dc1-2c2cfcb24e73-kube-api-access-k8lwv\") pod \"cert-manager-operator-controller-manager-55c444bf86-5dt79\" (UID: \"3035f933-a8bc-4e54-9dc1-2c2cfcb24e73\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" Jul 10 17:46:44.852401 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.852327 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3035f933-a8bc-4e54-9dc1-2c2cfcb24e73-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5dt79\" (UID: \"3035f933-a8bc-4e54-9dc1-2c2cfcb24e73\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" Jul 10 17:46:44.953073 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.953049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3035f933-a8bc-4e54-9dc1-2c2cfcb24e73-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5dt79\" (UID: \"3035f933-a8bc-4e54-9dc1-2c2cfcb24e73\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" Jul 10 17:46:44.953189 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.953098 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8lwv\" (UniqueName: \"kubernetes.io/projected/3035f933-a8bc-4e54-9dc1-2c2cfcb24e73-kube-api-access-k8lwv\") pod \"cert-manager-operator-controller-manager-55c444bf86-5dt79\" (UID: \"3035f933-a8bc-4e54-9dc1-2c2cfcb24e73\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" Jul 10 17:46:44.953481 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.953462 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3035f933-a8bc-4e54-9dc1-2c2cfcb24e73-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5dt79\" (UID: \"3035f933-a8bc-4e54-9dc1-2c2cfcb24e73\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" Jul 10 17:46:44.964066 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:44.964044 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8lwv\" (UniqueName: \"kubernetes.io/projected/3035f933-a8bc-4e54-9dc1-2c2cfcb24e73-kube-api-access-k8lwv\") pod \"cert-manager-operator-controller-manager-55c444bf86-5dt79\" (UID: \"3035f933-a8bc-4e54-9dc1-2c2cfcb24e73\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" Jul 10 17:46:45.105166 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:45.105133 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" Jul 10 17:46:45.236984 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:45.236950 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79"] Jul 10 17:46:45.242609 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:46:45.242575 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3035f933_a8bc_4e54_9dc1_2c2cfcb24e73.slice/crio-8a9980f025424b3ec686e5f0c5567ef7838059aec860a9fccb8f3ff40432388b WatchSource:0}: Error finding container 8a9980f025424b3ec686e5f0c5567ef7838059aec860a9fccb8f3ff40432388b: Status 404 returned error can't find the container with id 8a9980f025424b3ec686e5f0c5567ef7838059aec860a9fccb8f3ff40432388b Jul 10 17:46:45.430131 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:45.430052 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" event={"ID":"3035f933-a8bc-4e54-9dc1-2c2cfcb24e73","Type":"ContainerStarted","Data":"8a9980f025424b3ec686e5f0c5567ef7838059aec860a9fccb8f3ff40432388b"} Jul 10 17:46:47.440572 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:47.440535 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" event={"ID":"3035f933-a8bc-4e54-9dc1-2c2cfcb24e73","Type":"ContainerStarted","Data":"846a12b064ad280864844f8a624d50adc1fccafe92a4f1203b8b3fc03b9e739d"} Jul 10 17:46:47.480066 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:47.480019 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5dt79" podStartSLOduration=2.013227752 podStartE2EDuration="3.48000536s" totalImagesPullingTime="1.466777608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:46:47.477134902 +0000 UTC m=+485.304783672" watchObservedRunningTime="2026-07-10 17:46:47.48000536 +0000 UTC m=+485.307654127" Jul 10 17:46:48.274131 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.274096 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5"] Jul 10 17:46:48.277761 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.277739 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.280498 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.280474 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 17:46:48.280790 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.280738 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cj2k7\"" Jul 10 17:46:48.281230 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.281212 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 17:46:48.296353 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.296331 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5"] Jul 10 17:46:48.379023 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.378963 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbzlx\" (UniqueName: \"kubernetes.io/projected/7802b947-f8db-4de9-99bf-3a84216b0647-kube-api-access-hbzlx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.379023 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.379020 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.379219 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.379042 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.480167 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.480138 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbzlx\" (UniqueName: \"kubernetes.io/projected/7802b947-f8db-4de9-99bf-3a84216b0647-kube-api-access-hbzlx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.480501 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.480171 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.480501 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.480191 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.480575 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.480506 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.480575 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.480528 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.491013 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.490989 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbzlx\" (UniqueName: \"kubernetes.io/projected/7802b947-f8db-4de9-99bf-3a84216b0647-kube-api-access-hbzlx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.587340 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.587285 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:48.720268 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:48.720245 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5"] Jul 10 17:46:48.722881 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:46:48.722859 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7802b947_f8db_4de9_99bf_3a84216b0647.slice/crio-0241cebcf1d6df6378a20ea147689c16e3c6af4ce1db6e20d80b40b3b9f44178 WatchSource:0}: Error finding container 0241cebcf1d6df6378a20ea147689c16e3c6af4ce1db6e20d80b40b3b9f44178: Status 404 returned error can't find the container with id 0241cebcf1d6df6378a20ea147689c16e3c6af4ce1db6e20d80b40b3b9f44178 Jul 10 17:46:49.451098 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:49.451047 2568 generic.go:356] "Generic (PLEG): container finished" podID="7802b947-f8db-4de9-99bf-3a84216b0647" containerID="fed89f26d6c976f20f64e210180b9d3eec377d22cdd8fd04bc82f5d5a8b0f290" exitCode=0 Jul 10 17:46:49.451295 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:49.451141 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" event={"ID":"7802b947-f8db-4de9-99bf-3a84216b0647","Type":"ContainerDied","Data":"fed89f26d6c976f20f64e210180b9d3eec377d22cdd8fd04bc82f5d5a8b0f290"} Jul 10 17:46:49.451295 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:49.451194 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" event={"ID":"7802b947-f8db-4de9-99bf-3a84216b0647","Type":"ContainerStarted","Data":"0241cebcf1d6df6378a20ea147689c16e3c6af4ce1db6e20d80b40b3b9f44178"} Jul 10 17:46:50.823015 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:50.822959 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pm6q6"] Jul 10 17:46:50.827462 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:50.827432 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:50.830049 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:50.830021 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 17:46:50.830193 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:50.830082 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4k4ks\"" Jul 10 17:46:50.830890 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:50.830870 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 17:46:50.836597 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:50.836561 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pm6q6"] Jul 10 17:46:50.902893 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:50.902858 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfqv4\" (UniqueName: \"kubernetes.io/projected/6208ab59-80fe-481f-a848-8e80e0eaf4fa-kube-api-access-xfqv4\") pod \"cert-manager-webhook-6486ffd997-pm6q6\" (UID: \"6208ab59-80fe-481f-a848-8e80e0eaf4fa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:50.903086 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:50.902922 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6208ab59-80fe-481f-a848-8e80e0eaf4fa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pm6q6\" (UID: \"6208ab59-80fe-481f-a848-8e80e0eaf4fa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:51.003573 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:51.003537 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfqv4\" (UniqueName: \"kubernetes.io/projected/6208ab59-80fe-481f-a848-8e80e0eaf4fa-kube-api-access-xfqv4\") pod \"cert-manager-webhook-6486ffd997-pm6q6\" (UID: \"6208ab59-80fe-481f-a848-8e80e0eaf4fa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:51.003753 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:51.003605 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6208ab59-80fe-481f-a848-8e80e0eaf4fa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pm6q6\" (UID: \"6208ab59-80fe-481f-a848-8e80e0eaf4fa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:51.012527 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:51.012435 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfqv4\" (UniqueName: \"kubernetes.io/projected/6208ab59-80fe-481f-a848-8e80e0eaf4fa-kube-api-access-xfqv4\") pod \"cert-manager-webhook-6486ffd997-pm6q6\" (UID: \"6208ab59-80fe-481f-a848-8e80e0eaf4fa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:51.012659 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:51.012606 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6208ab59-80fe-481f-a848-8e80e0eaf4fa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pm6q6\" (UID: \"6208ab59-80fe-481f-a848-8e80e0eaf4fa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:51.150823 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:51.150792 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:51.627348 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:51.627323 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pm6q6"] Jul 10 17:46:51.629037 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:46:51.629008 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6208ab59_80fe_481f_a848_8e80e0eaf4fa.slice/crio-af960a80447e520bed33a7c293da2bfd24320e3e73e519c1587881200a26fdf8 WatchSource:0}: Error finding container af960a80447e520bed33a7c293da2bfd24320e3e73e519c1587881200a26fdf8: Status 404 returned error can't find the container with id af960a80447e520bed33a7c293da2bfd24320e3e73e519c1587881200a26fdf8 Jul 10 17:46:52.469481 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:52.469432 2568 generic.go:356] "Generic (PLEG): container finished" podID="7802b947-f8db-4de9-99bf-3a84216b0647" containerID="67a10b78f44223640ce46c301a64056296fdcf06f45483d4445c00290d7d5e86" exitCode=0 Jul 10 17:46:52.469935 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:52.469527 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" event={"ID":"7802b947-f8db-4de9-99bf-3a84216b0647","Type":"ContainerDied","Data":"67a10b78f44223640ce46c301a64056296fdcf06f45483d4445c00290d7d5e86"} Jul 10 17:46:52.470778 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:52.470755 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" event={"ID":"6208ab59-80fe-481f-a848-8e80e0eaf4fa","Type":"ContainerStarted","Data":"af960a80447e520bed33a7c293da2bfd24320e3e73e519c1587881200a26fdf8"} Jul 10 17:46:53.478450 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:53.478418 2568 generic.go:356] "Generic (PLEG): container finished" podID="7802b947-f8db-4de9-99bf-3a84216b0647" containerID="7bbcbe474ea9a58516eaecda33831567201757579e3aefc9f356cca7aa2e8999" exitCode=0 Jul 10 17:46:53.478839 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:53.478481 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" event={"ID":"7802b947-f8db-4de9-99bf-3a84216b0647","Type":"ContainerDied","Data":"7bbcbe474ea9a58516eaecda33831567201757579e3aefc9f356cca7aa2e8999"} Jul 10 17:46:54.485394 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.485305 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" event={"ID":"6208ab59-80fe-481f-a848-8e80e0eaf4fa","Type":"ContainerStarted","Data":"053976d4e4b259cfb7dd13f9e539598bf57b4f1515a37a22db0f800cd4b0a6e9"} Jul 10 17:46:54.485771 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.485564 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:46:54.503819 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.503771 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" podStartSLOduration=2.01068424 podStartE2EDuration="4.503755811s" totalImagesPullingTime="2.493071571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:46:54.502671033 +0000 UTC m=+492.330319832" watchObservedRunningTime="2026-07-10 17:46:54.503755811 +0000 UTC m=+492.331404579" Jul 10 17:46:54.618890 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.618861 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:54.736502 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.736427 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbzlx\" (UniqueName: \"kubernetes.io/projected/7802b947-f8db-4de9-99bf-3a84216b0647-kube-api-access-hbzlx\") pod \"7802b947-f8db-4de9-99bf-3a84216b0647\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " Jul 10 17:46:54.736502 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.736459 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-util\") pod \"7802b947-f8db-4de9-99bf-3a84216b0647\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " Jul 10 17:46:54.736502 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.736484 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-bundle\") pod \"7802b947-f8db-4de9-99bf-3a84216b0647\" (UID: \"7802b947-f8db-4de9-99bf-3a84216b0647\") " Jul 10 17:46:54.736913 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.736890 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-bundle" (OuterVolumeSpecName: "bundle") pod "7802b947-f8db-4de9-99bf-3a84216b0647" (UID: "7802b947-f8db-4de9-99bf-3a84216b0647"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:46:54.738558 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.738537 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7802b947-f8db-4de9-99bf-3a84216b0647-kube-api-access-hbzlx" (OuterVolumeSpecName: "kube-api-access-hbzlx") pod "7802b947-f8db-4de9-99bf-3a84216b0647" (UID: "7802b947-f8db-4de9-99bf-3a84216b0647"). InnerVolumeSpecName "kube-api-access-hbzlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:46:54.740857 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.740831 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-util" (OuterVolumeSpecName: "util") pod "7802b947-f8db-4de9-99bf-3a84216b0647" (UID: "7802b947-f8db-4de9-99bf-3a84216b0647"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:46:54.837418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.837396 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbzlx\" (UniqueName: \"kubernetes.io/projected/7802b947-f8db-4de9-99bf-3a84216b0647-kube-api-access-hbzlx\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:46:54.837418 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.837419 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-util\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:46:54.837571 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:54.837429 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7802b947-f8db-4de9-99bf-3a84216b0647-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:46:55.491009 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:55.490957 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" event={"ID":"7802b947-f8db-4de9-99bf-3a84216b0647","Type":"ContainerDied","Data":"0241cebcf1d6df6378a20ea147689c16e3c6af4ce1db6e20d80b40b3b9f44178"} Jul 10 17:46:55.491009 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:55.491017 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0241cebcf1d6df6378a20ea147689c16e3c6af4ce1db6e20d80b40b3b9f44178" Jul 10 17:46:55.491516 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:55.491014 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9c4h5" Jul 10 17:46:58.742394 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.742359 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bdlms"] Jul 10 17:46:58.742840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.742666 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7802b947-f8db-4de9-99bf-3a84216b0647" containerName="util" Jul 10 17:46:58.742840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.742676 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7802b947-f8db-4de9-99bf-3a84216b0647" containerName="util" Jul 10 17:46:58.742840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.742684 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7802b947-f8db-4de9-99bf-3a84216b0647" containerName="extract" Jul 10 17:46:58.742840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.742689 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7802b947-f8db-4de9-99bf-3a84216b0647" containerName="extract" Jul 10 17:46:58.742840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.742696 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7802b947-f8db-4de9-99bf-3a84216b0647" containerName="pull" Jul 10 17:46:58.742840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.742701 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7802b947-f8db-4de9-99bf-3a84216b0647" containerName="pull" Jul 10 17:46:58.742840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.742754 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7802b947-f8db-4de9-99bf-3a84216b0647" containerName="extract" Jul 10 17:46:58.746516 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.746498 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bdlms" Jul 10 17:46:58.748711 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.748687 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-gnj9n\"" Jul 10 17:46:58.755796 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.755776 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bdlms"] Jul 10 17:46:58.869840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.869805 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f72ee08-559e-4538-b2d8-68bd82e92376-bound-sa-token\") pod \"cert-manager-55b4d954d5-bdlms\" (UID: \"2f72ee08-559e-4538-b2d8-68bd82e92376\") " pod="cert-manager/cert-manager-55b4d954d5-bdlms" Jul 10 17:46:58.869840 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.869838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqrcp\" (UniqueName: \"kubernetes.io/projected/2f72ee08-559e-4538-b2d8-68bd82e92376-kube-api-access-dqrcp\") pod \"cert-manager-55b4d954d5-bdlms\" (UID: \"2f72ee08-559e-4538-b2d8-68bd82e92376\") " pod="cert-manager/cert-manager-55b4d954d5-bdlms" Jul 10 17:46:58.970955 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.970925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f72ee08-559e-4538-b2d8-68bd82e92376-bound-sa-token\") pod \"cert-manager-55b4d954d5-bdlms\" (UID: \"2f72ee08-559e-4538-b2d8-68bd82e92376\") " pod="cert-manager/cert-manager-55b4d954d5-bdlms" Jul 10 17:46:58.970955 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.970955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqrcp\" (UniqueName: \"kubernetes.io/projected/2f72ee08-559e-4538-b2d8-68bd82e92376-kube-api-access-dqrcp\") pod \"cert-manager-55b4d954d5-bdlms\" (UID: \"2f72ee08-559e-4538-b2d8-68bd82e92376\") " pod="cert-manager/cert-manager-55b4d954d5-bdlms" Jul 10 17:46:58.979329 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.979299 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqrcp\" (UniqueName: \"kubernetes.io/projected/2f72ee08-559e-4538-b2d8-68bd82e92376-kube-api-access-dqrcp\") pod \"cert-manager-55b4d954d5-bdlms\" (UID: \"2f72ee08-559e-4538-b2d8-68bd82e92376\") " pod="cert-manager/cert-manager-55b4d954d5-bdlms" Jul 10 17:46:58.979428 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:58.979330 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2f72ee08-559e-4538-b2d8-68bd82e92376-bound-sa-token\") pod \"cert-manager-55b4d954d5-bdlms\" (UID: \"2f72ee08-559e-4538-b2d8-68bd82e92376\") " pod="cert-manager/cert-manager-55b4d954d5-bdlms" Jul 10 17:46:59.056107 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:59.056045 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-bdlms" Jul 10 17:46:59.187460 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:59.187425 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-bdlms"] Jul 10 17:46:59.190560 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:46:59.190531 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f72ee08_559e_4538_b2d8_68bd82e92376.slice/crio-b2e2aaed5e6573b7dca0bb4e8829a3c069b15c65431f9e69c817d01da18d0b9b WatchSource:0}: Error finding container b2e2aaed5e6573b7dca0bb4e8829a3c069b15c65431f9e69c817d01da18d0b9b: Status 404 returned error can't find the container with id b2e2aaed5e6573b7dca0bb4e8829a3c069b15c65431f9e69c817d01da18d0b9b Jul 10 17:46:59.509319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:59.509274 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bdlms" event={"ID":"2f72ee08-559e-4538-b2d8-68bd82e92376","Type":"ContainerStarted","Data":"651a66f01ffb5e27ab8aa73a44126daf576ac8f77ee42ab24e789b252ed5de6e"} Jul 10 17:46:59.509319 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:59.509322 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-bdlms" event={"ID":"2f72ee08-559e-4538-b2d8-68bd82e92376","Type":"ContainerStarted","Data":"b2e2aaed5e6573b7dca0bb4e8829a3c069b15c65431f9e69c817d01da18d0b9b"} Jul 10 17:46:59.526061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:46:59.526018 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-bdlms" podStartSLOduration=1.526004264 podStartE2EDuration="1.526004264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:46:59.524106019 +0000 UTC m=+497.351754790" watchObservedRunningTime="2026-07-10 17:46:59.526004264 +0000 UTC m=+497.353653029" Jul 10 17:47:00.493816 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:00.493785 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-pm6q6" Jul 10 17:47:28.438815 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.438773 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr"] Jul 10 17:47:28.442136 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.442114 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.444807 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.444788 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 17:47:28.445722 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.445691 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 17:47:28.445801 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.445702 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cj2k7\"" Jul 10 17:47:28.454132 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.454108 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr"] Jul 10 17:47:28.499334 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.499306 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k27pw\" (UniqueName: \"kubernetes.io/projected/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-kube-api-access-k27pw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.499518 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.499395 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.499518 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.499440 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.600422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.600393 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.600581 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.600430 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k27pw\" (UniqueName: \"kubernetes.io/projected/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-kube-api-access-k27pw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.600581 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.600473 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.600856 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.600834 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.600856 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.600850 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.609663 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.609631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k27pw\" (UniqueName: \"kubernetes.io/projected/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-kube-api-access-k27pw\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.751775 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.751679 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:28.889005 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:28.888981 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr"] Jul 10 17:47:28.890821 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:47:28.890789 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ac26e9_3cc0_464c_b6ca_7b36620f28e4.slice/crio-372c014f854fab993f2eddd0b18b25a5d7dadb1b0628c49608457d68190be73d WatchSource:0}: Error finding container 372c014f854fab993f2eddd0b18b25a5d7dadb1b0628c49608457d68190be73d: Status 404 returned error can't find the container with id 372c014f854fab993f2eddd0b18b25a5d7dadb1b0628c49608457d68190be73d Jul 10 17:47:29.636107 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:29.636078 2568 generic.go:356] "Generic (PLEG): container finished" podID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerID="65e7e1039f3e16ce5e9e67b12fe83b0067d4e464fadb26667e1e9911b94ad05c" exitCode=0 Jul 10 17:47:29.636502 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:29.636141 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" event={"ID":"86ac26e9-3cc0-464c-b6ca-7b36620f28e4","Type":"ContainerDied","Data":"65e7e1039f3e16ce5e9e67b12fe83b0067d4e464fadb26667e1e9911b94ad05c"} Jul 10 17:47:29.636502 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:29.636176 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" event={"ID":"86ac26e9-3cc0-464c-b6ca-7b36620f28e4","Type":"ContainerStarted","Data":"372c014f854fab993f2eddd0b18b25a5d7dadb1b0628c49608457d68190be73d"} Jul 10 17:47:30.647416 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:30.647381 2568 generic.go:356] "Generic (PLEG): container finished" podID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerID="01864beae696c70bf7e2ff3ee73417cf25518cf4c1a446b8222329432d60ec7a" exitCode=0 Jul 10 17:47:30.647833 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:30.647468 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" event={"ID":"86ac26e9-3cc0-464c-b6ca-7b36620f28e4","Type":"ContainerDied","Data":"01864beae696c70bf7e2ff3ee73417cf25518cf4c1a446b8222329432d60ec7a"} Jul 10 17:47:31.654176 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:31.654114 2568 generic.go:356] "Generic (PLEG): container finished" podID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerID="cbcaa1e191f99904b1fe2a204fb1357d4e31a76dc2b670951725474ff27d4ab2" exitCode=0 Jul 10 17:47:31.654615 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:31.654200 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" event={"ID":"86ac26e9-3cc0-464c-b6ca-7b36620f28e4","Type":"ContainerDied","Data":"cbcaa1e191f99904b1fe2a204fb1357d4e31a76dc2b670951725474ff27d4ab2"} Jul 10 17:47:32.793739 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:32.793712 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:32.937991 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:32.937893 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k27pw\" (UniqueName: \"kubernetes.io/projected/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-kube-api-access-k27pw\") pod \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " Jul 10 17:47:32.937991 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:32.937934 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-bundle\") pod \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " Jul 10 17:47:32.937991 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:32.937963 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-util\") pod \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\" (UID: \"86ac26e9-3cc0-464c-b6ca-7b36620f28e4\") " Jul 10 17:47:32.938798 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:32.938771 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-bundle" (OuterVolumeSpecName: "bundle") pod "86ac26e9-3cc0-464c-b6ca-7b36620f28e4" (UID: "86ac26e9-3cc0-464c-b6ca-7b36620f28e4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:47:32.940064 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:32.940034 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-kube-api-access-k27pw" (OuterVolumeSpecName: "kube-api-access-k27pw") pod "86ac26e9-3cc0-464c-b6ca-7b36620f28e4" (UID: "86ac26e9-3cc0-464c-b6ca-7b36620f28e4"). InnerVolumeSpecName "kube-api-access-k27pw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:47:32.946491 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:32.946468 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-util" (OuterVolumeSpecName: "util") pod "86ac26e9-3cc0-464c-b6ca-7b36620f28e4" (UID: "86ac26e9-3cc0-464c-b6ca-7b36620f28e4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:47:33.038471 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.038438 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k27pw\" (UniqueName: \"kubernetes.io/projected/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-kube-api-access-k27pw\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:47:33.038471 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.038465 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:47:33.038471 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.038477 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/86ac26e9-3cc0-464c-b6ca-7b36620f28e4-util\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:47:33.216080 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.216003 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw"] Jul 10 17:47:33.216374 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.216361 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerName="pull" Jul 10 17:47:33.216422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.216377 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerName="pull" Jul 10 17:47:33.216422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.216389 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerName="extract" Jul 10 17:47:33.216422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.216394 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerName="extract" Jul 10 17:47:33.216422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.216404 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerName="util" Jul 10 17:47:33.216422 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.216410 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerName="util" Jul 10 17:47:33.216562 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.216460 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="86ac26e9-3cc0-464c-b6ca-7b36620f28e4" containerName="extract" Jul 10 17:47:33.219590 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.219575 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.222205 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.222182 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 17:47:33.222332 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.222232 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 17:47:33.223106 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.223086 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 17:47:33.223106 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.223093 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cgnpf\"" Jul 10 17:47:33.223246 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.223139 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 17:47:33.223246 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.223172 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 17:47:33.231645 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.231626 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw"] Jul 10 17:47:33.340805 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.340777 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-metrics-cert\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.340955 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.340814 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f6s2\" (UniqueName: \"kubernetes.io/projected/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-kube-api-access-4f6s2\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.340955 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.340846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-manager-config\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.340955 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.340902 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-cert\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.441872 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.441845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-manager-config\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.442007 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.441878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-cert\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.442007 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.441927 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-metrics-cert\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.442007 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.441986 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f6s2\" (UniqueName: \"kubernetes.io/projected/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-kube-api-access-4f6s2\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.442638 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.442585 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-manager-config\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.444547 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.444527 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-metrics-cert\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.444630 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.444574 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-cert\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.451223 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.451203 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f6s2\" (UniqueName: \"kubernetes.io/projected/52efb99b-99ac-4b7e-95f9-d4e4f3b85322-kube-api-access-4f6s2\") pod \"lws-controller-manager-7659478ddc-mkqfw\" (UID: \"52efb99b-99ac-4b7e-95f9-d4e4f3b85322\") " pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.528845 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.528787 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:33.658785 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.658752 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw"] Jul 10 17:47:33.662292 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:47:33.662269 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52efb99b_99ac_4b7e_95f9_d4e4f3b85322.slice/crio-e570bd1576e869cce3e5607ccbbc39ebb3284a574b19f969dcf4eed9be10cf06 WatchSource:0}: Error finding container e570bd1576e869cce3e5607ccbbc39ebb3284a574b19f969dcf4eed9be10cf06: Status 404 returned error can't find the container with id e570bd1576e869cce3e5607ccbbc39ebb3284a574b19f969dcf4eed9be10cf06 Jul 10 17:47:33.664557 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.664539 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" Jul 10 17:47:33.664682 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.664555 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afv28xr" event={"ID":"86ac26e9-3cc0-464c-b6ca-7b36620f28e4","Type":"ContainerDied","Data":"372c014f854fab993f2eddd0b18b25a5d7dadb1b0628c49608457d68190be73d"} Jul 10 17:47:33.664682 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:33.664599 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="372c014f854fab993f2eddd0b18b25a5d7dadb1b0628c49608457d68190be73d" Jul 10 17:47:34.670766 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:34.670726 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" event={"ID":"52efb99b-99ac-4b7e-95f9-d4e4f3b85322","Type":"ContainerStarted","Data":"e570bd1576e869cce3e5607ccbbc39ebb3284a574b19f969dcf4eed9be10cf06"} Jul 10 17:47:35.678044 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:35.678007 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" event={"ID":"52efb99b-99ac-4b7e-95f9-d4e4f3b85322","Type":"ContainerStarted","Data":"aa52c696ba1a90457e793ac1cb35bd6ce897a797933a98b9dc95cf725e3b076f"} Jul 10 17:47:35.678461 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:35.678137 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:35.693685 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:35.693634 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" podStartSLOduration=1.129990273 podStartE2EDuration="2.693618073s" totalImagesPullingTime="1.5636278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:47:35.693240345 +0000 UTC m=+533.520889117" watchObservedRunningTime="2026-07-10 17:47:35.693618073 +0000 UTC m=+533.521266845" Jul 10 17:47:43.131302 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.131262 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb"] Jul 10 17:47:43.135019 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.135000 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.138080 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.138060 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 17:47:43.138345 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.138326 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7wd7s\"" Jul 10 17:47:43.138542 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.138522 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 17:47:43.138542 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.138533 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 17:47:43.138651 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.138531 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 17:47:43.149827 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.149808 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb"] Jul 10 17:47:43.226765 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.226725 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-manifests\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.226924 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.226835 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48gjw\" (UniqueName: \"kubernetes.io/projected/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-kube-api-access-48gjw\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.226924 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.226861 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.226924 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.226890 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-tmp\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.226924 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.226911 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.327347 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.327308 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-tmp\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.327538 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.327355 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.327538 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.327398 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-manifests\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.327538 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.327469 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48gjw\" (UniqueName: \"kubernetes.io/projected/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-kube-api-access-48gjw\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.327538 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.327497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.327812 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.327783 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-manifests\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.329852 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.329817 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-tmp\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.330058 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.330034 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-webhook-cert\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.330148 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.330132 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.341097 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.341077 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48gjw\" (UniqueName: \"kubernetes.io/projected/cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d-kube-api-access-48gjw\") pod \"opendatahub-operator-controller-manager-dfdbc665d-tk6xb\" (UID: \"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d\") " pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.444713 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.444641 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:43.582231 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.582198 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb"] Jul 10 17:47:43.585864 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:47:43.585836 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8e04e5_d4a1_41fe_bbd8_2bc73684f46d.slice/crio-9d61ca74b095ea1d54689350d864ddcd7496133524ae264feec4b79cd05cd2ea WatchSource:0}: Error finding container 9d61ca74b095ea1d54689350d864ddcd7496133524ae264feec4b79cd05cd2ea: Status 404 returned error can't find the container with id 9d61ca74b095ea1d54689350d864ddcd7496133524ae264feec4b79cd05cd2ea Jul 10 17:47:43.714334 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:43.714242 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" event={"ID":"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d","Type":"ContainerStarted","Data":"9d61ca74b095ea1d54689350d864ddcd7496133524ae264feec4b79cd05cd2ea"} Jul 10 17:47:46.684875 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:46.684847 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7659478ddc-mkqfw" Jul 10 17:47:46.731420 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:46.731381 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" event={"ID":"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d","Type":"ContainerStarted","Data":"2e7a5edae9aefa0cee161e4778bbbc6b2cdb750d7aa935ec3f4c3e3253f4a33b"} Jul 10 17:47:47.736908 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:47.736874 2568 generic.go:356] "Generic (PLEG): container finished" podID="cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d" containerID="2e7a5edae9aefa0cee161e4778bbbc6b2cdb750d7aa935ec3f4c3e3253f4a33b" exitCode=0 Jul 10 17:47:47.737215 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:47.736921 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" event={"ID":"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d","Type":"ContainerDied","Data":"2e7a5edae9aefa0cee161e4778bbbc6b2cdb750d7aa935ec3f4c3e3253f4a33b"} Jul 10 17:47:48.744543 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:48.744509 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" event={"ID":"cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d","Type":"ContainerStarted","Data":"40f7cf8c4fbd0359020064ebb9ddf4b53ced2919f37df7c9b1073852d87d3127"} Jul 10 17:47:48.744912 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:48.744628 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:47:48.765750 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:48.765693 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" podStartSLOduration=1.431240237 podStartE2EDuration="5.765673869s" totalImagesPullingTime="3.296717798s" totalInitContainerRuntime="1.037715834s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:47:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:47:48.763792804 +0000 UTC m=+546.591441573" watchObservedRunningTime="2026-07-10 17:47:48.765673869 +0000 UTC m=+546.593322637" Jul 10 17:47:59.752657 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:47:59.752620 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-dfdbc665d-tk6xb" Jul 10 17:48:09.927576 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:09.927542 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw"] Jul 10 17:48:09.930942 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:09.930926 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:09.936453 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:09.936426 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 17:48:09.936453 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:09.936427 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 17:48:09.936639 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:09.936555 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 17:48:09.936949 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:09.936908 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cttkq\"" Jul 10 17:48:09.937087 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:09.937075 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 17:48:09.949886 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:09.949838 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw"] Jul 10 17:48:10.046722 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.046691 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.046722 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.046723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec0a9fc2-7081-44c2-b255-b7f49203c594-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.046925 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.046749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.046925 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.046770 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfpzs\" (UniqueName: \"kubernetes.io/projected/ec0a9fc2-7081-44c2-b255-b7f49203c594-kube-api-access-hfpzs\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.046925 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.046821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.046925 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.046836 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.046925 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.046851 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.148160 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.148131 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.148160 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.148162 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.148381 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.148180 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.148381 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.148206 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.148381 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.148307 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec0a9fc2-7081-44c2-b255-b7f49203c594-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.148381 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.148367 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.148593 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.148396 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfpzs\" (UniqueName: \"kubernetes.io/projected/ec0a9fc2-7081-44c2-b255-b7f49203c594-kube-api-access-hfpzs\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.148864 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.148837 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.150760 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.150730 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec0a9fc2-7081-44c2-b255-b7f49203c594-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.150863 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.150768 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.150935 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.150879 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.150935 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.150899 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ec0a9fc2-7081-44c2-b255-b7f49203c594-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.155763 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.155739 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec0a9fc2-7081-44c2-b255-b7f49203c594-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.155983 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.155949 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfpzs\" (UniqueName: \"kubernetes.io/projected/ec0a9fc2-7081-44c2-b255-b7f49203c594-kube-api-access-hfpzs\") pod \"istiod-openshift-gateway-94bb8fbfd-tr8vw\" (UID: \"ec0a9fc2-7081-44c2-b255-b7f49203c594\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.239853 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.239786 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:10.398081 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:48:10.398053 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec0a9fc2_7081_44c2_b255_b7f49203c594.slice/crio-a598f60387ff0c1a92aa821640746584db7435f5a2ff7fef1782519fc5ab6e53 WatchSource:0}: Error finding container a598f60387ff0c1a92aa821640746584db7435f5a2ff7fef1782519fc5ab6e53: Status 404 returned error can't find the container with id a598f60387ff0c1a92aa821640746584db7435f5a2ff7fef1782519fc5ab6e53 Jul 10 17:48:10.407214 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.407190 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw"] Jul 10 17:48:10.845413 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:10.845373 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" event={"ID":"ec0a9fc2-7081-44c2-b255-b7f49203c594","Type":"ContainerStarted","Data":"a598f60387ff0c1a92aa821640746584db7435f5a2ff7fef1782519fc5ab6e53"} Jul 10 17:48:14.185626 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:14.185580 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 10 17:48:14.185913 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:14.185673 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 10 17:48:14.873115 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:14.873077 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" event={"ID":"ec0a9fc2-7081-44c2-b255-b7f49203c594","Type":"ContainerStarted","Data":"b59d20be3ae5134021a469da1fcad4c392049923c187f52ea42bbf235d35342a"} Jul 10 17:48:14.873346 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:14.873182 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:14.899498 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:14.899435 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" podStartSLOduration=2.114403066 podStartE2EDuration="5.899414362s" totalImagesPullingTime="3.785011296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:48:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:48:14.898387541 +0000 UTC m=+572.726036397" watchObservedRunningTime="2026-07-10 17:48:14.899414362 +0000 UTC m=+572.727063132" Jul 10 17:48:15.879156 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:15.879129 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-tr8vw" Jul 10 17:48:46.281958 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:46.281927 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:48:46.284510 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:48:46.282643 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:49:19.198388 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.198355 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf"] Jul 10 17:49:19.202104 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.202087 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.204500 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.204473 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n84lr\"" Jul 10 17:49:19.204500 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.204497 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 17:49:19.204690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.204515 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 17:49:19.209779 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.209753 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf"] Jul 10 17:49:19.357060 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.357022 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b42mw\" (UniqueName: \"kubernetes.io/projected/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-kube-api-access-b42mw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.357245 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.357102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.357245 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.357135 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.457700 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.457619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42mw\" (UniqueName: \"kubernetes.io/projected/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-kube-api-access-b42mw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.457700 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.457683 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.457909 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.457714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.458080 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.458062 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.458165 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.458126 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.466796 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.466764 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b42mw\" (UniqueName: \"kubernetes.io/projected/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-kube-api-access-b42mw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.512617 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.512589 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:19.600727 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.599391 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk"] Jul 10 17:49:19.603141 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.603118 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.612921 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.612897 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk"] Jul 10 17:49:19.761387 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.761294 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hs7t\" (UniqueName: \"kubernetes.io/projected/c782e8c3-54ee-4785-9876-71162727063b-kube-api-access-2hs7t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.761387 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.761358 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.761612 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.761407 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.856038 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.856014 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf"] Jul 10 17:49:19.857741 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:49:19.857715 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120ec37c_c555_4ae9_b6c1_5af55f2e33f6.slice/crio-95f3cc38e68fc3d1f5988b1c7d1a96f0dd11914ea7d614f1f25a01d7ed734fc1 WatchSource:0}: Error finding container 95f3cc38e68fc3d1f5988b1c7d1a96f0dd11914ea7d614f1f25a01d7ed734fc1: Status 404 returned error can't find the container with id 95f3cc38e68fc3d1f5988b1c7d1a96f0dd11914ea7d614f1f25a01d7ed734fc1 Jul 10 17:49:19.862817 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.862792 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.862939 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.862821 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.862939 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.862877 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hs7t\" (UniqueName: \"kubernetes.io/projected/c782e8c3-54ee-4785-9876-71162727063b-kube-api-access-2hs7t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.863174 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.863154 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.863239 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.863199 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.875689 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.875663 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hs7t\" (UniqueName: \"kubernetes.io/projected/c782e8c3-54ee-4785-9876-71162727063b-kube-api-access-2hs7t\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.915790 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.915771 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:19.997921 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:19.997891 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8"] Jul 10 17:49:20.001473 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.001451 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.013023 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.012906 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8"] Jul 10 17:49:20.052591 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.052569 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk"] Jul 10 17:49:20.054591 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:49:20.054564 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc782e8c3_54ee_4785_9876_71162727063b.slice/crio-7cbd554ab191f40eff441d5f352eb8882244931ac943d15e784be34f180f9a5a WatchSource:0}: Error finding container 7cbd554ab191f40eff441d5f352eb8882244931ac943d15e784be34f180f9a5a: Status 404 returned error can't find the container with id 7cbd554ab191f40eff441d5f352eb8882244931ac943d15e784be34f180f9a5a Jul 10 17:49:20.165834 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.165802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbzxl\" (UniqueName: \"kubernetes.io/projected/6ccaa546-0100-415f-95bb-07d3b2f0391a-kube-api-access-wbzxl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.165957 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.165883 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.165957 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.165920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.172041 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.172012 2568 generic.go:356] "Generic (PLEG): container finished" podID="c782e8c3-54ee-4785-9876-71162727063b" containerID="bc25736ac9043c3869da78a824fde56d39cec520f524bc1e928bb8bbf378ead3" exitCode=0 Jul 10 17:49:20.172161 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.172100 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" event={"ID":"c782e8c3-54ee-4785-9876-71162727063b","Type":"ContainerDied","Data":"bc25736ac9043c3869da78a824fde56d39cec520f524bc1e928bb8bbf378ead3"} Jul 10 17:49:20.172229 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.172158 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" event={"ID":"c782e8c3-54ee-4785-9876-71162727063b","Type":"ContainerStarted","Data":"7cbd554ab191f40eff441d5f352eb8882244931ac943d15e784be34f180f9a5a"} Jul 10 17:49:20.172630 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.172610 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 17:49:20.173793 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.173764 2568 generic.go:356] "Generic (PLEG): container finished" podID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerID="1ab612d7962097ad9d6019169b58749c34f4ce527369b854b013cc6bdd957d8d" exitCode=0 Jul 10 17:49:20.173885 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.173805 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" event={"ID":"120ec37c-c555-4ae9-b6c1-5af55f2e33f6","Type":"ContainerDied","Data":"1ab612d7962097ad9d6019169b58749c34f4ce527369b854b013cc6bdd957d8d"} Jul 10 17:49:20.173885 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.173842 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" event={"ID":"120ec37c-c555-4ae9-b6c1-5af55f2e33f6","Type":"ContainerStarted","Data":"95f3cc38e68fc3d1f5988b1c7d1a96f0dd11914ea7d614f1f25a01d7ed734fc1"} Jul 10 17:49:20.266998 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.266881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbzxl\" (UniqueName: \"kubernetes.io/projected/6ccaa546-0100-415f-95bb-07d3b2f0391a-kube-api-access-wbzxl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.266998 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.266933 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.267496 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.267105 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.267496 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.267325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.267496 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.267394 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.276381 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.276349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbzxl\" (UniqueName: \"kubernetes.io/projected/6ccaa546-0100-415f-95bb-07d3b2f0391a-kube-api-access-wbzxl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.314849 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.314822 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:20.408135 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.408106 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx"] Jul 10 17:49:20.412638 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.412615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.419528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.419500 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx"] Jul 10 17:49:20.450477 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.450320 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8"] Jul 10 17:49:20.452573 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:49:20.452539 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ccaa546_0100_415f_95bb_07d3b2f0391a.slice/crio-04769c4ecfec5fab845d47d74bb366a047873b93da0522eca5fd68004791f59b WatchSource:0}: Error finding container 04769c4ecfec5fab845d47d74bb366a047873b93da0522eca5fd68004791f59b: Status 404 returned error can't find the container with id 04769c4ecfec5fab845d47d74bb366a047873b93da0522eca5fd68004791f59b Jul 10 17:49:20.569742 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.569720 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.569851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.569752 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.569851 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.569777 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdfsp\" (UniqueName: \"kubernetes.io/projected/de8c0e60-699f-45bd-86be-24e61e9cb454-kube-api-access-gdfsp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.670617 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.670587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.670743 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.670623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.670743 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.670649 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdfsp\" (UniqueName: \"kubernetes.io/projected/de8c0e60-699f-45bd-86be-24e61e9cb454-kube-api-access-gdfsp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.671028 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.671004 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.671120 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.671039 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.679400 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.679375 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdfsp\" (UniqueName: \"kubernetes.io/projected/de8c0e60-699f-45bd-86be-24e61e9cb454-kube-api-access-gdfsp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.724578 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.724548 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:20.859200 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:20.859177 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx"] Jul 10 17:49:20.860931 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:49:20.860907 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde8c0e60_699f_45bd_86be_24e61e9cb454.slice/crio-dad592e4d783fcaf499971346edd13133447bbcfb2cc6d3382edd0cf6c9e7354 WatchSource:0}: Error finding container dad592e4d783fcaf499971346edd13133447bbcfb2cc6d3382edd0cf6c9e7354: Status 404 returned error can't find the container with id dad592e4d783fcaf499971346edd13133447bbcfb2cc6d3382edd0cf6c9e7354 Jul 10 17:49:21.180638 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.180611 2568 generic.go:356] "Generic (PLEG): container finished" podID="c782e8c3-54ee-4785-9876-71162727063b" containerID="e0f57e05dd94de2ba19f9a81859394faa79d10fd3bf229c76de3f41e4c6a186e" exitCode=0 Jul 10 17:49:21.180723 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.180695 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" event={"ID":"c782e8c3-54ee-4785-9876-71162727063b","Type":"ContainerDied","Data":"e0f57e05dd94de2ba19f9a81859394faa79d10fd3bf229c76de3f41e4c6a186e"} Jul 10 17:49:21.182715 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.182691 2568 generic.go:356] "Generic (PLEG): container finished" podID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerID="7d1497d98c16bd963c39b185f1da90302336562e9e00b297771cba0566bc48da" exitCode=0 Jul 10 17:49:21.182803 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.182774 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" event={"ID":"120ec37c-c555-4ae9-b6c1-5af55f2e33f6","Type":"ContainerDied","Data":"7d1497d98c16bd963c39b185f1da90302336562e9e00b297771cba0566bc48da"} Jul 10 17:49:21.184494 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.184472 2568 generic.go:356] "Generic (PLEG): container finished" podID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerID="c52a3b1932961f4a78c4cd48753cb6d1cb677b8cba1c68ce76281656744de82e" exitCode=0 Jul 10 17:49:21.184563 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.184543 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" event={"ID":"de8c0e60-699f-45bd-86be-24e61e9cb454","Type":"ContainerDied","Data":"c52a3b1932961f4a78c4cd48753cb6d1cb677b8cba1c68ce76281656744de82e"} Jul 10 17:49:21.184639 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.184623 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" event={"ID":"de8c0e60-699f-45bd-86be-24e61e9cb454","Type":"ContainerStarted","Data":"dad592e4d783fcaf499971346edd13133447bbcfb2cc6d3382edd0cf6c9e7354"} Jul 10 17:49:21.186128 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.186106 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerID="ce417c2a0f6656d3ff6b351598ba44ddd71ca10296405983a85d14053ad290b1" exitCode=0 Jul 10 17:49:21.186220 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.186137 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" event={"ID":"6ccaa546-0100-415f-95bb-07d3b2f0391a","Type":"ContainerDied","Data":"ce417c2a0f6656d3ff6b351598ba44ddd71ca10296405983a85d14053ad290b1"} Jul 10 17:49:21.186220 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:21.186171 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" event={"ID":"6ccaa546-0100-415f-95bb-07d3b2f0391a","Type":"ContainerStarted","Data":"04769c4ecfec5fab845d47d74bb366a047873b93da0522eca5fd68004791f59b"} Jul 10 17:49:21.381890 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:49:21.381860 2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120ec37c_c555_4ae9_b6c1_5af55f2e33f6.slice/crio-40b089b067c8426f3c21353f4c5280f5fd77d06c598c09630f034446a160a9ad.scope\": RecentStats: unable to find data in memory cache]" Jul 10 17:49:22.193093 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:22.193068 2568 generic.go:356] "Generic (PLEG): container finished" podID="c782e8c3-54ee-4785-9876-71162727063b" containerID="0ea2e401a489cfce2e3303b54ff9b37526b0e4f90a7777ea71fdde8907602dde" exitCode=0 Jul 10 17:49:22.193184 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:22.193149 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" event={"ID":"c782e8c3-54ee-4785-9876-71162727063b","Type":"ContainerDied","Data":"0ea2e401a489cfce2e3303b54ff9b37526b0e4f90a7777ea71fdde8907602dde"} Jul 10 17:49:22.195100 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:22.195080 2568 generic.go:356] "Generic (PLEG): container finished" podID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerID="40b089b067c8426f3c21353f4c5280f5fd77d06c598c09630f034446a160a9ad" exitCode=0 Jul 10 17:49:22.195205 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:22.195172 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" event={"ID":"120ec37c-c555-4ae9-b6c1-5af55f2e33f6","Type":"ContainerDied","Data":"40b089b067c8426f3c21353f4c5280f5fd77d06c598c09630f034446a160a9ad"} Jul 10 17:49:22.196782 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:22.196760 2568 generic.go:356] "Generic (PLEG): container finished" podID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerID="285051ec93e00661b1f3a5a7f19321fe4793977f1fbc5fe1e6db352bcd39119b" exitCode=0 Jul 10 17:49:22.196860 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:22.196789 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" event={"ID":"de8c0e60-699f-45bd-86be-24e61e9cb454","Type":"ContainerDied","Data":"285051ec93e00661b1f3a5a7f19321fe4793977f1fbc5fe1e6db352bcd39119b"} Jul 10 17:49:22.198641 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:22.198622 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerID="ba289ad6f7ecb727ffb98b633a9a2fba6c99830fee79a0aaea38b563a9117c82" exitCode=0 Jul 10 17:49:22.198721 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:22.198654 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" event={"ID":"6ccaa546-0100-415f-95bb-07d3b2f0391a","Type":"ContainerDied","Data":"ba289ad6f7ecb727ffb98b633a9a2fba6c99830fee79a0aaea38b563a9117c82"} Jul 10 17:49:23.205512 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.205481 2568 generic.go:356] "Generic (PLEG): container finished" podID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerID="70be4efa0dc47c007e3bfca252ae1404247cd2d07a9821300d677ec6f2f4d4e8" exitCode=0 Jul 10 17:49:23.205989 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.205550 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" event={"ID":"de8c0e60-699f-45bd-86be-24e61e9cb454","Type":"ContainerDied","Data":"70be4efa0dc47c007e3bfca252ae1404247cd2d07a9821300d677ec6f2f4d4e8"} Jul 10 17:49:23.207604 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.207576 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerID="498b09366d4e905dca3c85b4942f9b39f29271116a2bfffc9584025e7b0dc083" exitCode=0 Jul 10 17:49:23.207729 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.207657 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" event={"ID":"6ccaa546-0100-415f-95bb-07d3b2f0391a","Type":"ContainerDied","Data":"498b09366d4e905dca3c85b4942f9b39f29271116a2bfffc9584025e7b0dc083"} Jul 10 17:49:23.363455 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.363436 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:23.366242 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.366222 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:23.493415 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.493330 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-util\") pod \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " Jul 10 17:49:23.493415 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.493360 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-bundle\") pod \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " Jul 10 17:49:23.493415 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.493401 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b42mw\" (UniqueName: \"kubernetes.io/projected/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-kube-api-access-b42mw\") pod \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\" (UID: \"120ec37c-c555-4ae9-b6c1-5af55f2e33f6\") " Jul 10 17:49:23.493681 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.493434 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-bundle\") pod \"c782e8c3-54ee-4785-9876-71162727063b\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " Jul 10 17:49:23.493681 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.493469 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hs7t\" (UniqueName: \"kubernetes.io/projected/c782e8c3-54ee-4785-9876-71162727063b-kube-api-access-2hs7t\") pod \"c782e8c3-54ee-4785-9876-71162727063b\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " Jul 10 17:49:23.493681 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.493523 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-util\") pod \"c782e8c3-54ee-4785-9876-71162727063b\" (UID: \"c782e8c3-54ee-4785-9876-71162727063b\") " Jul 10 17:49:23.494209 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.494151 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-bundle" (OuterVolumeSpecName: "bundle") pod "c782e8c3-54ee-4785-9876-71162727063b" (UID: "c782e8c3-54ee-4785-9876-71162727063b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:49:23.494360 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.494203 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-bundle" (OuterVolumeSpecName: "bundle") pod "120ec37c-c555-4ae9-b6c1-5af55f2e33f6" (UID: "120ec37c-c555-4ae9-b6c1-5af55f2e33f6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:49:23.495862 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.495842 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c782e8c3-54ee-4785-9876-71162727063b-kube-api-access-2hs7t" (OuterVolumeSpecName: "kube-api-access-2hs7t") pod "c782e8c3-54ee-4785-9876-71162727063b" (UID: "c782e8c3-54ee-4785-9876-71162727063b"). InnerVolumeSpecName "kube-api-access-2hs7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:49:23.496086 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.496064 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-kube-api-access-b42mw" (OuterVolumeSpecName: "kube-api-access-b42mw") pod "120ec37c-c555-4ae9-b6c1-5af55f2e33f6" (UID: "120ec37c-c555-4ae9-b6c1-5af55f2e33f6"). InnerVolumeSpecName "kube-api-access-b42mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:49:23.500120 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.500100 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-util" (OuterVolumeSpecName: "util") pod "c782e8c3-54ee-4785-9876-71162727063b" (UID: "c782e8c3-54ee-4785-9876-71162727063b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:49:23.501692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.501671 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-util" (OuterVolumeSpecName: "util") pod "120ec37c-c555-4ae9-b6c1-5af55f2e33f6" (UID: "120ec37c-c555-4ae9-b6c1-5af55f2e33f6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:49:23.594317 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.594297 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-util\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:23.594317 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.594317 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:23.594443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.594326 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b42mw\" (UniqueName: \"kubernetes.io/projected/120ec37c-c555-4ae9-b6c1-5af55f2e33f6-kube-api-access-b42mw\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:23.594443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.594335 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:23.594443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.594343 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hs7t\" (UniqueName: \"kubernetes.io/projected/c782e8c3-54ee-4785-9876-71162727063b-kube-api-access-2hs7t\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:23.594443 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:23.594351 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c782e8c3-54ee-4785-9876-71162727063b-util\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:24.215022 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.214993 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" Jul 10 17:49:24.215022 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.215006 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk" event={"ID":"c782e8c3-54ee-4785-9876-71162727063b","Type":"ContainerDied","Data":"7cbd554ab191f40eff441d5f352eb8882244931ac943d15e784be34f180f9a5a"} Jul 10 17:49:24.215485 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.215083 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cbd554ab191f40eff441d5f352eb8882244931ac943d15e784be34f180f9a5a" Jul 10 17:49:24.216864 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.216845 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" Jul 10 17:49:24.217011 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.216863 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf" event={"ID":"120ec37c-c555-4ae9-b6c1-5af55f2e33f6","Type":"ContainerDied","Data":"95f3cc38e68fc3d1f5988b1c7d1a96f0dd11914ea7d614f1f25a01d7ed734fc1"} Jul 10 17:49:24.217011 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.216904 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95f3cc38e68fc3d1f5988b1c7d1a96f0dd11914ea7d614f1f25a01d7ed734fc1" Jul 10 17:49:24.351727 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.351699 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:24.376168 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.376145 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:24.401064 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.401029 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-util\") pod \"6ccaa546-0100-415f-95bb-07d3b2f0391a\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " Jul 10 17:49:24.401181 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.401076 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-util\") pod \"de8c0e60-699f-45bd-86be-24e61e9cb454\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " Jul 10 17:49:24.401181 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.401103 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-bundle\") pod \"6ccaa546-0100-415f-95bb-07d3b2f0391a\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " Jul 10 17:49:24.401181 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.401136 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdfsp\" (UniqueName: \"kubernetes.io/projected/de8c0e60-699f-45bd-86be-24e61e9cb454-kube-api-access-gdfsp\") pod \"de8c0e60-699f-45bd-86be-24e61e9cb454\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " Jul 10 17:49:24.401342 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.401184 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbzxl\" (UniqueName: \"kubernetes.io/projected/6ccaa546-0100-415f-95bb-07d3b2f0391a-kube-api-access-wbzxl\") pod \"6ccaa546-0100-415f-95bb-07d3b2f0391a\" (UID: \"6ccaa546-0100-415f-95bb-07d3b2f0391a\") " Jul 10 17:49:24.401342 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.401235 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-bundle\") pod \"de8c0e60-699f-45bd-86be-24e61e9cb454\" (UID: \"de8c0e60-699f-45bd-86be-24e61e9cb454\") " Jul 10 17:49:24.401625 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.401599 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-bundle" (OuterVolumeSpecName: "bundle") pod "6ccaa546-0100-415f-95bb-07d3b2f0391a" (UID: "6ccaa546-0100-415f-95bb-07d3b2f0391a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:49:24.402084 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.402052 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-bundle" (OuterVolumeSpecName: "bundle") pod "de8c0e60-699f-45bd-86be-24e61e9cb454" (UID: "de8c0e60-699f-45bd-86be-24e61e9cb454"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:49:24.403782 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.403751 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ccaa546-0100-415f-95bb-07d3b2f0391a-kube-api-access-wbzxl" (OuterVolumeSpecName: "kube-api-access-wbzxl") pod "6ccaa546-0100-415f-95bb-07d3b2f0391a" (UID: "6ccaa546-0100-415f-95bb-07d3b2f0391a"). InnerVolumeSpecName "kube-api-access-wbzxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:49:24.403782 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.403770 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de8c0e60-699f-45bd-86be-24e61e9cb454-kube-api-access-gdfsp" (OuterVolumeSpecName: "kube-api-access-gdfsp") pod "de8c0e60-699f-45bd-86be-24e61e9cb454" (UID: "de8c0e60-699f-45bd-86be-24e61e9cb454"). InnerVolumeSpecName "kube-api-access-gdfsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:49:24.407272 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.407244 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-util" (OuterVolumeSpecName: "util") pod "6ccaa546-0100-415f-95bb-07d3b2f0391a" (UID: "6ccaa546-0100-415f-95bb-07d3b2f0391a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:49:24.410385 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.410344 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-util" (OuterVolumeSpecName: "util") pod "de8c0e60-699f-45bd-86be-24e61e9cb454" (UID: "de8c0e60-699f-45bd-86be-24e61e9cb454"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:49:24.502548 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.502524 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:24.502548 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.502546 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-util\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:24.502676 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.502554 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de8c0e60-699f-45bd-86be-24e61e9cb454-util\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:24.502676 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.502561 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ccaa546-0100-415f-95bb-07d3b2f0391a-bundle\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:24.502676 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.502569 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdfsp\" (UniqueName: \"kubernetes.io/projected/de8c0e60-699f-45bd-86be-24e61e9cb454-kube-api-access-gdfsp\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:24.502676 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:24.502579 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbzxl\" (UniqueName: \"kubernetes.io/projected/6ccaa546-0100-415f-95bb-07d3b2f0391a-kube-api-access-wbzxl\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:49:25.223713 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:25.223676 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" event={"ID":"de8c0e60-699f-45bd-86be-24e61e9cb454","Type":"ContainerDied","Data":"dad592e4d783fcaf499971346edd13133447bbcfb2cc6d3382edd0cf6c9e7354"} Jul 10 17:49:25.224174 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:25.223726 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dad592e4d783fcaf499971346edd13133447bbcfb2cc6d3382edd0cf6c9e7354" Jul 10 17:49:25.224174 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:25.223685 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx" Jul 10 17:49:25.225944 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:25.225887 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" Jul 10 17:49:25.226075 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:25.225888 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8" event={"ID":"6ccaa546-0100-415f-95bb-07d3b2f0391a","Type":"ContainerDied","Data":"04769c4ecfec5fab845d47d74bb366a047873b93da0522eca5fd68004791f59b"} Jul 10 17:49:25.226075 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:49:25.226018 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04769c4ecfec5fab845d47d74bb366a047873b93da0522eca5fd68004791f59b" Jul 10 17:50:26.067626 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067591 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-px2nx"] Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067900 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerName="pull" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067911 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerName="pull" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067920 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerName="pull" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067925 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerName="pull" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067936 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerName="util" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067942 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerName="util" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067948 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerName="extract" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067952 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerName="extract" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067959 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c782e8c3-54ee-4785-9876-71162727063b" containerName="util" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067980 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c782e8c3-54ee-4785-9876-71162727063b" containerName="util" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067993 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c782e8c3-54ee-4785-9876-71162727063b" containerName="extract" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.067998 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c782e8c3-54ee-4785-9876-71162727063b" containerName="extract" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068009 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerName="extract" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068014 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerName="extract" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068020 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerName="util" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068026 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerName="util" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068031 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerName="util" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068036 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerName="util" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068041 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c782e8c3-54ee-4785-9876-71162727063b" containerName="pull" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068046 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c782e8c3-54ee-4785-9876-71162727063b" containerName="pull" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068053 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerName="extract" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068059 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerName="extract" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068066 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerName="pull" Jul 10 17:50:26.068061 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068070 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerName="pull" Jul 10 17:50:26.068732 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068122 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c782e8c3-54ee-4785-9876-71162727063b" containerName="extract" Jul 10 17:50:26.068732 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068130 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="120ec37c-c555-4ae9-b6c1-5af55f2e33f6" containerName="extract" Jul 10 17:50:26.068732 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068136 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ccaa546-0100-415f-95bb-07d3b2f0391a" containerName="extract" Jul 10 17:50:26.068732 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.068143 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="de8c0e60-699f-45bd-86be-24e61e9cb454" containerName="extract" Jul 10 17:50:26.071147 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.071130 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:26.073528 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.073502 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 17:50:26.073660 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.073551 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 17:50:26.073791 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.073776 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n84lr\"" Jul 10 17:50:26.075912 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.075894 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 17:50:26.081104 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.081078 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-px2nx"] Jul 10 17:50:26.107587 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.107559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpz9b\" (UniqueName: \"kubernetes.io/projected/ec621237-a54d-42f3-b1bb-60f82c93689a-kube-api-access-vpz9b\") pod \"limitador-limitador-64c94fd64f-px2nx\" (UID: \"ec621237-a54d-42f3-b1bb-60f82c93689a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:26.107749 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.107613 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec621237-a54d-42f3-b1bb-60f82c93689a-config-file\") pod \"limitador-limitador-64c94fd64f-px2nx\" (UID: \"ec621237-a54d-42f3-b1bb-60f82c93689a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:26.160060 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.160028 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-px2nx"] Jul 10 17:50:26.208266 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.208218 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec621237-a54d-42f3-b1bb-60f82c93689a-config-file\") pod \"limitador-limitador-64c94fd64f-px2nx\" (UID: \"ec621237-a54d-42f3-b1bb-60f82c93689a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:26.208432 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.208321 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpz9b\" (UniqueName: \"kubernetes.io/projected/ec621237-a54d-42f3-b1bb-60f82c93689a-kube-api-access-vpz9b\") pod \"limitador-limitador-64c94fd64f-px2nx\" (UID: \"ec621237-a54d-42f3-b1bb-60f82c93689a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:26.208830 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.208812 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec621237-a54d-42f3-b1bb-60f82c93689a-config-file\") pod \"limitador-limitador-64c94fd64f-px2nx\" (UID: \"ec621237-a54d-42f3-b1bb-60f82c93689a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:26.217184 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.217157 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpz9b\" (UniqueName: \"kubernetes.io/projected/ec621237-a54d-42f3-b1bb-60f82c93689a-kube-api-access-vpz9b\") pod \"limitador-limitador-64c94fd64f-px2nx\" (UID: \"ec621237-a54d-42f3-b1bb-60f82c93689a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:26.382259 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.382227 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:26.516029 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.516005 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-px2nx"] Jul 10 17:50:26.518017 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:50:26.517993 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec621237_a54d_42f3_b1bb_60f82c93689a.slice/crio-c5289fe54d4bc4100d53646465386c19792af2b6be0e98a7dab640c7c350f26a WatchSource:0}: Error finding container c5289fe54d4bc4100d53646465386c19792af2b6be0e98a7dab640c7c350f26a: Status 404 returned error can't find the container with id c5289fe54d4bc4100d53646465386c19792af2b6be0e98a7dab640c7c350f26a Jul 10 17:50:26.533307 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:26.533277 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" event={"ID":"ec621237-a54d-42f3-b1bb-60f82c93689a","Type":"ContainerStarted","Data":"c5289fe54d4bc4100d53646465386c19792af2b6be0e98a7dab640c7c350f26a"} Jul 10 17:50:29.553344 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:29.553255 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" event={"ID":"ec621237-a54d-42f3-b1bb-60f82c93689a","Type":"ContainerStarted","Data":"b78872cb306f535669d41fb620fad26c0f5dac3e611bbc55c6457f6e20a99fde"} Jul 10 17:50:29.553718 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:29.553384 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:29.590272 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:29.590212 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" podStartSLOduration=0.846623026 podStartE2EDuration="3.59019333s" totalImagesPullingTime="2.743570304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:50:29.590096218 +0000 UTC m=+707.417744997" watchObservedRunningTime="2026-07-10 17:50:29.59019333 +0000 UTC m=+707.417842101" Jul 10 17:50:40.560500 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:40.560468 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:43.184265 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.184224 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-px2nx"] Jul 10 17:50:43.184833 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.184425 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" podUID="ec621237-a54d-42f3-b1bb-60f82c93689a" containerName="limitador" containerID="cri-o://b78872cb306f535669d41fb620fad26c0f5dac3e611bbc55c6457f6e20a99fde" gracePeriod=30 Jul 10 17:50:43.625093 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.625062 2568 generic.go:356] "Generic (PLEG): container finished" podID="ec621237-a54d-42f3-b1bb-60f82c93689a" containerID="b78872cb306f535669d41fb620fad26c0f5dac3e611bbc55c6457f6e20a99fde" exitCode=0 Jul 10 17:50:43.625256 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.625138 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" event={"ID":"ec621237-a54d-42f3-b1bb-60f82c93689a","Type":"ContainerDied","Data":"b78872cb306f535669d41fb620fad26c0f5dac3e611bbc55c6457f6e20a99fde"} Jul 10 17:50:43.737959 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.737935 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:43.865518 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.865470 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec621237-a54d-42f3-b1bb-60f82c93689a-config-file\") pod \"ec621237-a54d-42f3-b1bb-60f82c93689a\" (UID: \"ec621237-a54d-42f3-b1bb-60f82c93689a\") " Jul 10 17:50:43.865692 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.865600 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpz9b\" (UniqueName: \"kubernetes.io/projected/ec621237-a54d-42f3-b1bb-60f82c93689a-kube-api-access-vpz9b\") pod \"ec621237-a54d-42f3-b1bb-60f82c93689a\" (UID: \"ec621237-a54d-42f3-b1bb-60f82c93689a\") " Jul 10 17:50:43.865909 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.865883 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec621237-a54d-42f3-b1bb-60f82c93689a-config-file" (OuterVolumeSpecName: "config-file") pod "ec621237-a54d-42f3-b1bb-60f82c93689a" (UID: "ec621237-a54d-42f3-b1bb-60f82c93689a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 17:50:43.868158 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.868128 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec621237-a54d-42f3-b1bb-60f82c93689a-kube-api-access-vpz9b" (OuterVolumeSpecName: "kube-api-access-vpz9b") pod "ec621237-a54d-42f3-b1bb-60f82c93689a" (UID: "ec621237-a54d-42f3-b1bb-60f82c93689a"). InnerVolumeSpecName "kube-api-access-vpz9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:50:43.967206 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.967176 2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ec621237-a54d-42f3-b1bb-60f82c93689a-config-file\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:50:43.967206 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:43.967201 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpz9b\" (UniqueName: \"kubernetes.io/projected/ec621237-a54d-42f3-b1bb-60f82c93689a-kube-api-access-vpz9b\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:50:44.630838 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:44.630804 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" Jul 10 17:50:44.631315 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:44.630803 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-px2nx" event={"ID":"ec621237-a54d-42f3-b1bb-60f82c93689a","Type":"ContainerDied","Data":"c5289fe54d4bc4100d53646465386c19792af2b6be0e98a7dab640c7c350f26a"} Jul 10 17:50:44.631315 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:44.631010 2568 scope.go:117] "RemoveContainer" containerID="b78872cb306f535669d41fb620fad26c0f5dac3e611bbc55c6457f6e20a99fde" Jul 10 17:50:44.652485 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:44.652447 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-px2nx"] Jul 10 17:50:44.654522 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:44.654500 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-px2nx"] Jul 10 17:50:44.756357 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:44.756330 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec621237-a54d-42f3-b1bb-60f82c93689a" path="/var/lib/kubelet/pods/ec621237-a54d-42f3-b1bb-60f82c93689a/volumes" Jul 10 17:50:47.759026 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.758996 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-782ns"] Jul 10 17:50:47.759464 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.759449 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec621237-a54d-42f3-b1bb-60f82c93689a" containerName="limitador" Jul 10 17:50:47.759509 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.759469 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec621237-a54d-42f3-b1bb-60f82c93689a" containerName="limitador" Jul 10 17:50:47.759597 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.759586 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec621237-a54d-42f3-b1bb-60f82c93689a" containerName="limitador" Jul 10 17:50:47.768212 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.768187 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:47.768899 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.768877 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-782ns"] Jul 10 17:50:47.770591 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.770568 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 17:50:47.770714 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.770568 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 17:50:47.771789 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.771768 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-pt99k\"" Jul 10 17:50:47.771935 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.771805 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 17:50:47.899076 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.899043 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ztxr\" (UniqueName: \"kubernetes.io/projected/1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a-kube-api-access-4ztxr\") pod \"postgres-69fb4594cf-782ns\" (UID: \"1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:47.899237 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:47.899108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a-data\") pod \"postgres-69fb4594cf-782ns\" (UID: \"1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:48.000442 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:48.000401 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ztxr\" (UniqueName: \"kubernetes.io/projected/1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a-kube-api-access-4ztxr\") pod \"postgres-69fb4594cf-782ns\" (UID: \"1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:48.000625 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:48.000480 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a-data\") pod \"postgres-69fb4594cf-782ns\" (UID: \"1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:48.000919 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:48.000896 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a-data\") pod \"postgres-69fb4594cf-782ns\" (UID: \"1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:48.008532 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:48.008504 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ztxr\" (UniqueName: \"kubernetes.io/projected/1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a-kube-api-access-4ztxr\") pod \"postgres-69fb4594cf-782ns\" (UID: \"1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:48.080265 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:48.080184 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:48.211591 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:48.211567 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-782ns"] Jul 10 17:50:48.213696 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:50:48.213664 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5c97eb_155a_4bdc_9dca_7506cb6e0d6a.slice/crio-de906396564e1788ef42f5cfbeb8e18c55c0cc873223499031ad5ffb03a471ab WatchSource:0}: Error finding container de906396564e1788ef42f5cfbeb8e18c55c0cc873223499031ad5ffb03a471ab: Status 404 returned error can't find the container with id de906396564e1788ef42f5cfbeb8e18c55c0cc873223499031ad5ffb03a471ab Jul 10 17:50:48.659203 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:48.659168 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" event={"ID":"1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a","Type":"ContainerStarted","Data":"de906396564e1788ef42f5cfbeb8e18c55c0cc873223499031ad5ffb03a471ab"} Jul 10 17:50:53.691374 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:53.691336 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" event={"ID":"1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a","Type":"ContainerStarted","Data":"5a1a4032b807444058e8b02e671eff86cda52be859d6f99e59e2675b6e299cf3"} Jul 10 17:50:53.691734 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:53.691406 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:50:53.707849 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:53.707800 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" podStartSLOduration=1.934683659 podStartE2EDuration="6.707786106s" totalImagesPullingTime="4.773102447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:50:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:50:53.705291927 +0000 UTC m=+731.532940705" watchObservedRunningTime="2026-07-10 17:50:53.707786106 +0000 UTC m=+731.535434873" Jul 10 17:50:59.727021 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:50:59.726956 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-782ns" Jul 10 17:51:06.603487 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.603456 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-548d4cf6bc-cssfk"] Jul 10 17:51:06.606848 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.606830 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:06.609447 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.609426 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 17:51:06.609447 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.609433 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 17:51:06.609615 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.609433 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mj6ll\"" Jul 10 17:51:06.617047 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.617025 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-548d4cf6bc-cssfk"] Jul 10 17:51:06.664252 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.664223 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p685\" (UniqueName: \"kubernetes.io/projected/fa28f80a-5977-4c03-acc6-ca0a890bd485-kube-api-access-5p685\") pod \"maas-controller-548d4cf6bc-cssfk\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:06.664408 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.664272 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert\") pod \"maas-controller-548d4cf6bc-cssfk\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:06.765009 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.764948 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert\") pod \"maas-controller-548d4cf6bc-cssfk\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:06.765210 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.765064 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p685\" (UniqueName: \"kubernetes.io/projected/fa28f80a-5977-4c03-acc6-ca0a890bd485-kube-api-access-5p685\") pod \"maas-controller-548d4cf6bc-cssfk\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:06.765210 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:51:06.765147 2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 10 17:51:06.765304 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:51:06.765213 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert podName:fa28f80a-5977-4c03-acc6-ca0a890bd485 nodeName:}" failed. No retries permitted until 2026-07-10 17:51:07.265196139 +0000 UTC m=+745.092844884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert") pod "maas-controller-548d4cf6bc-cssfk" (UID: "fa28f80a-5977-4c03-acc6-ca0a890bd485") : secret "maas-controller-webhook-cert" not found Jul 10 17:51:06.772650 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.772626 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56fbcc7474-jkckl"] Jul 10 17:51:06.776032 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.776011 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:06.777490 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.777470 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p685\" (UniqueName: \"kubernetes.io/projected/fa28f80a-5977-4c03-acc6-ca0a890bd485-kube-api-access-5p685\") pod \"maas-controller-548d4cf6bc-cssfk\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:06.784668 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.784646 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56fbcc7474-jkckl"] Jul 10 17:51:06.866452 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.866391 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66qxw\" (UniqueName: \"kubernetes.io/projected/81bc6416-5ea7-498f-a85a-61eff3c9be9c-kube-api-access-66qxw\") pod \"maas-controller-56fbcc7474-jkckl\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:06.866452 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.866429 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert\") pod \"maas-controller-56fbcc7474-jkckl\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:06.966809 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.966783 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66qxw\" (UniqueName: \"kubernetes.io/projected/81bc6416-5ea7-498f-a85a-61eff3c9be9c-kube-api-access-66qxw\") pod \"maas-controller-56fbcc7474-jkckl\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:06.966961 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.966818 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert\") pod \"maas-controller-56fbcc7474-jkckl\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:06.966961 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:51:06.966915 2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 10 17:51:06.967064 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:51:06.967008 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert podName:81bc6416-5ea7-498f-a85a-61eff3c9be9c nodeName:}" failed. No retries permitted until 2026-07-10 17:51:07.466960707 +0000 UTC m=+745.294609454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert") pod "maas-controller-56fbcc7474-jkckl" (UID: "81bc6416-5ea7-498f-a85a-61eff3c9be9c") : secret "maas-controller-webhook-cert" not found Jul 10 17:51:06.977095 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:06.977063 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66qxw\" (UniqueName: \"kubernetes.io/projected/81bc6416-5ea7-498f-a85a-61eff3c9be9c-kube-api-access-66qxw\") pod \"maas-controller-56fbcc7474-jkckl\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:07.268939 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.268845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert\") pod \"maas-controller-548d4cf6bc-cssfk\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:07.271450 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.271426 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert\") pod \"maas-controller-548d4cf6bc-cssfk\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:07.470227 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.470188 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert\") pod \"maas-controller-56fbcc7474-jkckl\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:07.472873 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.472844 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert\") pod \"maas-controller-56fbcc7474-jkckl\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:07.517706 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.517684 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:07.652067 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.651864 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-548d4cf6bc-cssfk"] Jul 10 17:51:07.654799 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:51:07.654768 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa28f80a_5977_4c03_acc6_ca0a890bd485.slice/crio-dd002af57c1530538368651a07efde6900071dec2b00c1263fc44f238277c8e1 WatchSource:0}: Error finding container dd002af57c1530538368651a07efde6900071dec2b00c1263fc44f238277c8e1: Status 404 returned error can't find the container with id dd002af57c1530538368651a07efde6900071dec2b00c1263fc44f238277c8e1 Jul 10 17:51:07.695079 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.695051 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:07.762200 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.762163 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" event={"ID":"fa28f80a-5977-4c03-acc6-ca0a890bd485","Type":"ContainerStarted","Data":"dd002af57c1530538368651a07efde6900071dec2b00c1263fc44f238277c8e1"} Jul 10 17:51:07.843947 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:07.843911 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56fbcc7474-jkckl"] Jul 10 17:51:07.846890 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:51:07.846860 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81bc6416_5ea7_498f_a85a_61eff3c9be9c.slice/crio-20809182c09a350dcd74c5a3dc517a6c4081c2fc6fe8de2e7773b11f47bf596f WatchSource:0}: Error finding container 20809182c09a350dcd74c5a3dc517a6c4081c2fc6fe8de2e7773b11f47bf596f: Status 404 returned error can't find the container with id 20809182c09a350dcd74c5a3dc517a6c4081c2fc6fe8de2e7773b11f47bf596f Jul 10 17:51:08.771474 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:08.771398 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56fbcc7474-jkckl" event={"ID":"81bc6416-5ea7-498f-a85a-61eff3c9be9c","Type":"ContainerStarted","Data":"20809182c09a350dcd74c5a3dc517a6c4081c2fc6fe8de2e7773b11f47bf596f"} Jul 10 17:51:10.784016 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:10.783951 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56fbcc7474-jkckl" event={"ID":"81bc6416-5ea7-498f-a85a-61eff3c9be9c","Type":"ContainerStarted","Data":"52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a"} Jul 10 17:51:10.784481 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:10.784145 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:10.785498 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:10.785471 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" event={"ID":"fa28f80a-5977-4c03-acc6-ca0a890bd485","Type":"ContainerStarted","Data":"fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280"} Jul 10 17:51:10.785641 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:10.785532 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:10.804360 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:10.804308 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56fbcc7474-jkckl" podStartSLOduration=2.850065647 podStartE2EDuration="4.804296388s" totalImagesPullingTime="1.954230741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:51:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:51:10.80214502 +0000 UTC m=+748.629793789" watchObservedRunningTime="2026-07-10 17:51:10.804296388 +0000 UTC m=+748.631945154" Jul 10 17:51:10.819487 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:10.819453 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" podStartSLOduration=2.663111136 podStartE2EDuration="4.819440675s" totalImagesPullingTime="2.156329539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:51:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:51:10.818834771 +0000 UTC m=+748.646483541" watchObservedRunningTime="2026-07-10 17:51:10.819440675 +0000 UTC m=+748.647089443" Jul 10 17:51:21.795438 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:21.795401 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:51:21.795904 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:21.795705 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:21.856473 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:21.856440 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-548d4cf6bc-cssfk"] Jul 10 17:51:21.856822 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:21.856788 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" podUID="fa28f80a-5977-4c03-acc6-ca0a890bd485" containerName="manager" containerID="cri-o://fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280" gracePeriod=30 Jul 10 17:51:22.096726 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.096698 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:22.194738 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.194715 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p685\" (UniqueName: \"kubernetes.io/projected/fa28f80a-5977-4c03-acc6-ca0a890bd485-kube-api-access-5p685\") pod \"fa28f80a-5977-4c03-acc6-ca0a890bd485\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " Jul 10 17:51:22.194906 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.194773 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert\") pod \"fa28f80a-5977-4c03-acc6-ca0a890bd485\" (UID: \"fa28f80a-5977-4c03-acc6-ca0a890bd485\") " Jul 10 17:51:22.197066 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.197035 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fa28f80a-5977-4c03-acc6-ca0a890bd485" (UID: "fa28f80a-5977-4c03-acc6-ca0a890bd485"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:51:22.197180 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.197043 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa28f80a-5977-4c03-acc6-ca0a890bd485-kube-api-access-5p685" (OuterVolumeSpecName: "kube-api-access-5p685") pod "fa28f80a-5977-4c03-acc6-ca0a890bd485" (UID: "fa28f80a-5977-4c03-acc6-ca0a890bd485"). InnerVolumeSpecName "kube-api-access-5p685". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:51:22.295578 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.295551 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa28f80a-5977-4c03-acc6-ca0a890bd485-webhook-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:51:22.295578 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.295576 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p685\" (UniqueName: \"kubernetes.io/projected/fa28f80a-5977-4c03-acc6-ca0a890bd485-kube-api-access-5p685\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:51:22.849791 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.849752 2568 generic.go:356] "Generic (PLEG): container finished" podID="fa28f80a-5977-4c03-acc6-ca0a890bd485" containerID="fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280" exitCode=0 Jul 10 17:51:22.849791 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.849792 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" Jul 10 17:51:22.850200 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.849831 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" event={"ID":"fa28f80a-5977-4c03-acc6-ca0a890bd485","Type":"ContainerDied","Data":"fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280"} Jul 10 17:51:22.850200 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.849884 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-548d4cf6bc-cssfk" event={"ID":"fa28f80a-5977-4c03-acc6-ca0a890bd485","Type":"ContainerDied","Data":"dd002af57c1530538368651a07efde6900071dec2b00c1263fc44f238277c8e1"} Jul 10 17:51:22.850200 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.849908 2568 scope.go:117] "RemoveContainer" containerID="fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280" Jul 10 17:51:22.858749 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.858600 2568 scope.go:117] "RemoveContainer" containerID="fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280" Jul 10 17:51:22.858873 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:51:22.858853 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280\": container with ID starting with fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280 not found: ID does not exist" containerID="fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280" Jul 10 17:51:22.858917 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.858884 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280"} err="failed to get container status \"fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280\": rpc error: code = NotFound desc = could not find container \"fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280\": container with ID starting with fedf6de294ba4547e08d52727ed2a80c89e3260fa82b625b12423b219b92b280 not found: ID does not exist" Jul 10 17:51:22.867435 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.867412 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-548d4cf6bc-cssfk"] Jul 10 17:51:22.869397 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:22.869380 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-548d4cf6bc-cssfk"] Jul 10 17:51:24.755820 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:24.755787 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa28f80a-5977-4c03-acc6-ca0a890bd485" path="/var/lib/kubelet/pods/fa28f80a-5977-4c03-acc6-ca0a890bd485/volumes" Jul 10 17:51:43.256621 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.256589 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-b96984f7d-kthtc"] Jul 10 17:51:43.256999 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.256922 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa28f80a-5977-4c03-acc6-ca0a890bd485" containerName="manager" Jul 10 17:51:43.256999 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.256932 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa28f80a-5977-4c03-acc6-ca0a890bd485" containerName="manager" Jul 10 17:51:43.257082 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.257003 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa28f80a-5977-4c03-acc6-ca0a890bd485" containerName="manager" Jul 10 17:51:43.263073 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.263053 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.265488 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.265471 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 10 17:51:43.265565 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.265547 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xd7f2\"" Jul 10 17:51:43.269940 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.269914 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b96984f7d-kthtc"] Jul 10 17:51:43.380549 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.380515 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a128c9-acd8-4c93-a3bd-624cd318de0f-maas-api-tls\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.380711 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.380604 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a128c9-acd8-4c93-a3bd-624cd318de0f-openshift-service-ca\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.380711 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.380635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llgps\" (UniqueName: \"kubernetes.io/projected/a4a128c9-acd8-4c93-a3bd-624cd318de0f-kube-api-access-llgps\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.481419 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.481348 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a128c9-acd8-4c93-a3bd-624cd318de0f-openshift-service-ca\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.481419 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.481417 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llgps\" (UniqueName: \"kubernetes.io/projected/a4a128c9-acd8-4c93-a3bd-624cd318de0f-kube-api-access-llgps\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.481693 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.481492 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a128c9-acd8-4c93-a3bd-624cd318de0f-maas-api-tls\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.481693 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:51:43.481669 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 10 17:51:43.481800 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:51:43.481757 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4a128c9-acd8-4c93-a3bd-624cd318de0f-maas-api-tls podName:a4a128c9-acd8-4c93-a3bd-624cd318de0f nodeName:}" failed. No retries permitted until 2026-07-10 17:51:43.981736383 +0000 UTC m=+781.809385141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4a128c9-acd8-4c93-a3bd-624cd318de0f-maas-api-tls") pod "maas-api-b96984f7d-kthtc" (UID: "a4a128c9-acd8-4c93-a3bd-624cd318de0f") : secret "maas-api-serving-cert" not found Jul 10 17:51:43.482220 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.482194 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4a128c9-acd8-4c93-a3bd-624cd318de0f-openshift-service-ca\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.490524 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.490502 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llgps\" (UniqueName: \"kubernetes.io/projected/a4a128c9-acd8-4c93-a3bd-624cd318de0f-kube-api-access-llgps\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.789660 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.789609 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bzllf"] Jul 10 17:51:43.794036 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.794021 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:43.796347 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.796326 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 17:51:43.796347 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.796343 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-js2vj\"" Jul 10 17:51:43.803765 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.803737 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bzllf"] Jul 10 17:51:43.886411 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.886375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b4w2\" (UniqueName: \"kubernetes.io/projected/9b099279-a74f-4ed1-b641-e5d60c4d037e-kube-api-access-9b4w2\") pod \"payload-processing-58f86c7798-bzllf\" (UID: \"9b099279-a74f-4ed1-b641-e5d60c4d037e\") " pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:43.886813 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.886765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b099279-a74f-4ed1-b641-e5d60c4d037e-plugins-config-volume\") pod \"payload-processing-58f86c7798-bzllf\" (UID: \"9b099279-a74f-4ed1-b641-e5d60c4d037e\") " pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:43.987573 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.987539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b099279-a74f-4ed1-b641-e5d60c4d037e-plugins-config-volume\") pod \"payload-processing-58f86c7798-bzllf\" (UID: \"9b099279-a74f-4ed1-b641-e5d60c4d037e\") " pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:43.987573 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.987640 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a128c9-acd8-4c93-a3bd-624cd318de0f-maas-api-tls\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.987573 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.987685 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b4w2\" (UniqueName: \"kubernetes.io/projected/9b099279-a74f-4ed1-b641-e5d60c4d037e-kube-api-access-9b4w2\") pod \"payload-processing-58f86c7798-bzllf\" (UID: \"9b099279-a74f-4ed1-b641-e5d60c4d037e\") " pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:43.988454 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.988432 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b099279-a74f-4ed1-b641-e5d60c4d037e-plugins-config-volume\") pod \"payload-processing-58f86c7798-bzllf\" (UID: \"9b099279-a74f-4ed1-b641-e5d60c4d037e\") " pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:43.990385 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.990365 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4a128c9-acd8-4c93-a3bd-624cd318de0f-maas-api-tls\") pod \"maas-api-b96984f7d-kthtc\" (UID: \"a4a128c9-acd8-4c93-a3bd-624cd318de0f\") " pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:43.995464 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:43.995439 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b4w2\" (UniqueName: \"kubernetes.io/projected/9b099279-a74f-4ed1-b641-e5d60c4d037e-kube-api-access-9b4w2\") pod \"payload-processing-58f86c7798-bzllf\" (UID: \"9b099279-a74f-4ed1-b641-e5d60c4d037e\") " pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:44.107121 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:44.107098 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:44.175208 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:44.174871 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:44.248278 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:44.248239 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bzllf"] Jul 10 17:51:44.251270 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:51:44.251236 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b099279_a74f_4ed1_b641_e5d60c4d037e.slice/crio-1a6d06f86f28bc33057fb93222959e47350083a51bc0b7d40705a1ce2559a7ea WatchSource:0}: Error finding container 1a6d06f86f28bc33057fb93222959e47350083a51bc0b7d40705a1ce2559a7ea: Status 404 returned error can't find the container with id 1a6d06f86f28bc33057fb93222959e47350083a51bc0b7d40705a1ce2559a7ea Jul 10 17:51:44.318891 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:44.318862 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-b96984f7d-kthtc"] Jul 10 17:51:44.320501 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:51:44.320474 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a128c9_acd8_4c93_a3bd_624cd318de0f.slice/crio-53666bb480f1a96c84e35392370f59589f4a98746b3d14be2effbb6e33026732 WatchSource:0}: Error finding container 53666bb480f1a96c84e35392370f59589f4a98746b3d14be2effbb6e33026732: Status 404 returned error can't find the container with id 53666bb480f1a96c84e35392370f59589f4a98746b3d14be2effbb6e33026732 Jul 10 17:51:44.963744 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:44.963707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-bzllf" event={"ID":"9b099279-a74f-4ed1-b641-e5d60c4d037e","Type":"ContainerStarted","Data":"1a6d06f86f28bc33057fb93222959e47350083a51bc0b7d40705a1ce2559a7ea"} Jul 10 17:51:44.965662 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:44.965630 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" event={"ID":"a4a128c9-acd8-4c93-a3bd-624cd318de0f","Type":"ContainerStarted","Data":"53666bb480f1a96c84e35392370f59589f4a98746b3d14be2effbb6e33026732"} Jul 10 17:51:46.981886 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:46.981841 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-bzllf" event={"ID":"9b099279-a74f-4ed1-b641-e5d60c4d037e","Type":"ContainerStarted","Data":"6b52c9354278cc82e0f3dd25a9c9c8f143825debc8468a44dd8fc7f7c1a85367"} Jul 10 17:51:46.982346 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:46.981916 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:51:46.983428 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:46.983407 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" event={"ID":"a4a128c9-acd8-4c93-a3bd-624cd318de0f","Type":"ContainerStarted","Data":"34ac06b931be03c6c826fd1f32b706c4ff0cbe1bfbc628887710018ecdee5d25"} Jul 10 17:51:46.983555 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:46.983536 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:46.996516 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:46.996470 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-bzllf" podStartSLOduration=2.293436047 podStartE2EDuration="3.996452762s" totalImagesPullingTime="1.703016715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:51:46.99590735 +0000 UTC m=+784.823556118" watchObservedRunningTime="2026-07-10 17:51:46.996452762 +0000 UTC m=+784.824101529" Jul 10 17:51:47.014301 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:47.014240 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" podStartSLOduration=2.37724409 podStartE2EDuration="4.014221341s" totalImagesPullingTime="1.636977251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:51:47.011499849 +0000 UTC m=+784.839148640" watchObservedRunningTime="2026-07-10 17:51:47.014221341 +0000 UTC m=+784.841870109" Jul 10 17:51:52.996645 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:52.996613 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-b96984f7d-kthtc" Jul 10 17:51:57.993385 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:51:57.993355 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-bzllf" Jul 10 17:52:00.245394 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.245357 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t"] Jul 10 17:52:00.250318 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.250293 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.253872 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.253851 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 17:52:00.254016 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.253851 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 17:52:00.254016 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.253921 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 10 17:52:00.254016 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.253927 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zxrkb\"" Jul 10 17:52:00.263256 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.263234 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t"] Jul 10 17:52:00.329930 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.329886 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gftwm\" (UniqueName: \"kubernetes.io/projected/d915f4f9-296c-408a-ba07-a982a7285962-kube-api-access-gftwm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.329930 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.329930 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.330154 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.329951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.330154 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.330033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.330154 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.330135 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.330252 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.330163 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d915f4f9-296c-408a-ba07-a982a7285962-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.430811 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.430774 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gftwm\" (UniqueName: \"kubernetes.io/projected/d915f4f9-296c-408a-ba07-a982a7285962-kube-api-access-gftwm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.431030 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.430815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.431030 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.430846 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.431030 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.430872 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.431030 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.430934 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.431030 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.430957 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d915f4f9-296c-408a-ba07-a982a7285962-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.431288 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.431258 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.431339 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.431300 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.431339 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.431318 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.433402 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.433382 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d915f4f9-296c-408a-ba07-a982a7285962-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.433574 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.433559 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d915f4f9-296c-408a-ba07-a982a7285962-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.438299 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.438281 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gftwm\" (UniqueName: \"kubernetes.io/projected/d915f4f9-296c-408a-ba07-a982a7285962-kube-api-access-gftwm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t\" (UID: \"d915f4f9-296c-408a-ba07-a982a7285962\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.562034 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.561924 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:00.700463 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:00.698049 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t"] Jul 10 17:52:00.704604 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:52:00.704569 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd915f4f9_296c_408a_ba07_a982a7285962.slice/crio-9588582cf8b6c7917044ccdc2ee274639620cf5c8fef0d80df971482db1d57d8 WatchSource:0}: Error finding container 9588582cf8b6c7917044ccdc2ee274639620cf5c8fef0d80df971482db1d57d8: Status 404 returned error can't find the container with id 9588582cf8b6c7917044ccdc2ee274639620cf5c8fef0d80df971482db1d57d8 Jul 10 17:52:01.065862 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:01.065809 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" event={"ID":"d915f4f9-296c-408a-ba07-a982a7285962","Type":"ContainerStarted","Data":"9588582cf8b6c7917044ccdc2ee274639620cf5c8fef0d80df971482db1d57d8"} Jul 10 17:52:02.072589 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:02.072544 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" event={"ID":"d915f4f9-296c-408a-ba07-a982a7285962","Type":"ContainerStarted","Data":"9568bd8ef3e17f421179ddd687d741760f3b39c2a12e862499d0e0432f5aea06"} Jul 10 17:52:02.091898 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:02.091842 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" podStartSLOduration=0.92252343 podStartE2EDuration="2.091826196s" totalImagesPullingTime="1.169302766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:52:02.088829544 +0000 UTC m=+799.916478313" watchObservedRunningTime="2026-07-10 17:52:02.091826196 +0000 UTC m=+799.919475026" Jul 10 17:52:10.563067 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:10.563027 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:10.563490 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:10.563124 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:10.575462 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:10.575438 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:11.131842 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:11.131817 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t" Jul 10 17:52:16.243719 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.243689 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5"] Jul 10 17:52:16.247473 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.247446 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.249958 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.249929 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 10 17:52:16.260152 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.260132 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5"] Jul 10 17:52:16.364885 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.364853 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.365074 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.364889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.365074 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.364916 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7x7h\" (UniqueName: \"kubernetes.io/projected/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-kube-api-access-v7x7h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.365178 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.365074 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.365178 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.365110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.365274 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.365190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.466586 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.466553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.466748 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.466609 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.466748 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.466627 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.466748 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.466643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7x7h\" (UniqueName: \"kubernetes.io/projected/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-kube-api-access-v7x7h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.466748 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.466698 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.466748 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.466718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.467578 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.467060 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.467578 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.467116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.467578 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.467146 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.469174 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.469143 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.469438 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.469422 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.474020 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.474002 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7x7h\" (UniqueName: \"kubernetes.io/projected/9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19-kube-api-access-v7x7h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hw6t5\" (UID: \"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.558169 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.558101 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:16.693690 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:16.693662 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5"] Jul 10 17:52:16.695279 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:52:16.695253 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a64dcf5_fb52_4eac_8f2f_c6c08e6f8f19.slice/crio-a99d72d83f2fec03cdadf5159445aab6709c30d2c649f20dd57f6cd200a07670 WatchSource:0}: Error finding container a99d72d83f2fec03cdadf5159445aab6709c30d2c649f20dd57f6cd200a07670: Status 404 returned error can't find the container with id a99d72d83f2fec03cdadf5159445aab6709c30d2c649f20dd57f6cd200a07670 Jul 10 17:52:17.153121 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:17.153022 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" event={"ID":"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19","Type":"ContainerStarted","Data":"59b8d6540f9c0d0417d96b1505dbbdc3a32a8bef44455b38d352a8ca1fdd1311"} Jul 10 17:52:17.153121 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:17.153084 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" event={"ID":"9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19","Type":"ContainerStarted","Data":"a99d72d83f2fec03cdadf5159445aab6709c30d2c649f20dd57f6cd200a07670"} Jul 10 17:52:17.171034 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:17.170962 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" podStartSLOduration=0.972842176 podStartE2EDuration="1.170946452s" totalImagesPullingTime="198.104276ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:52:17.169091003 +0000 UTC m=+814.996739770" watchObservedRunningTime="2026-07-10 17:52:17.170946452 +0000 UTC m=+814.998595242" Jul 10 17:52:26.558431 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:26.558396 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:26.559062 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:26.558475 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:26.571101 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:26.571076 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:52:27.223098 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:52:27.223064 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hw6t5" Jul 10 17:53:46.318536 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:53:46.318427 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:53:46.319884 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:53:46.318830 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:54:39.942699 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:39.942667 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56fbcc7474-jkckl"] Jul 10 17:54:39.943267 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:39.942912 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56fbcc7474-jkckl" podUID="81bc6416-5ea7-498f-a85a-61eff3c9be9c" containerName="manager" containerID="cri-o://52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a" gracePeriod=30 Jul 10 17:54:40.198326 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.198267 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:54:40.343543 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.343510 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert\") pod \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " Jul 10 17:54:40.343703 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.343644 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66qxw\" (UniqueName: \"kubernetes.io/projected/81bc6416-5ea7-498f-a85a-61eff3c9be9c-kube-api-access-66qxw\") pod \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\" (UID: \"81bc6416-5ea7-498f-a85a-61eff3c9be9c\") " Jul 10 17:54:40.345713 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.345671 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "81bc6416-5ea7-498f-a85a-61eff3c9be9c" (UID: "81bc6416-5ea7-498f-a85a-61eff3c9be9c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:54:40.345829 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.345787 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81bc6416-5ea7-498f-a85a-61eff3c9be9c-kube-api-access-66qxw" (OuterVolumeSpecName: "kube-api-access-66qxw") pod "81bc6416-5ea7-498f-a85a-61eff3c9be9c" (UID: "81bc6416-5ea7-498f-a85a-61eff3c9be9c"). InnerVolumeSpecName "kube-api-access-66qxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:54:40.444322 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.444296 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66qxw\" (UniqueName: \"kubernetes.io/projected/81bc6416-5ea7-498f-a85a-61eff3c9be9c-kube-api-access-66qxw\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:54:40.444322 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.444319 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81bc6416-5ea7-498f-a85a-61eff3c9be9c-webhook-cert\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 17:54:40.906314 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.906282 2568 generic.go:356] "Generic (PLEG): container finished" podID="81bc6416-5ea7-498f-a85a-61eff3c9be9c" containerID="52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a" exitCode=0 Jul 10 17:54:40.906466 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.906324 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56fbcc7474-jkckl" Jul 10 17:54:40.906466 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.906372 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56fbcc7474-jkckl" event={"ID":"81bc6416-5ea7-498f-a85a-61eff3c9be9c","Type":"ContainerDied","Data":"52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a"} Jul 10 17:54:40.906466 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.906426 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56fbcc7474-jkckl" event={"ID":"81bc6416-5ea7-498f-a85a-61eff3c9be9c","Type":"ContainerDied","Data":"20809182c09a350dcd74c5a3dc517a6c4081c2fc6fe8de2e7773b11f47bf596f"} Jul 10 17:54:40.906466 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.906452 2568 scope.go:117] "RemoveContainer" containerID="52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a" Jul 10 17:54:40.915612 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.915592 2568 scope.go:117] "RemoveContainer" containerID="52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a" Jul 10 17:54:40.915886 ip-10-0-131-229 kubenswrapper[2568]: E0710 17:54:40.915867 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a\": container with ID starting with 52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a not found: ID does not exist" containerID="52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a" Jul 10 17:54:40.915933 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.915896 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a"} err="failed to get container status \"52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a\": rpc error: code = NotFound desc = could not find container \"52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a\": container with ID starting with 52f57b2438227e2baa2d8b2c602f444d1a50153331c10c3701d19c0fd572751a not found: ID does not exist" Jul 10 17:54:40.924859 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.924838 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56fbcc7474-jkckl"] Jul 10 17:54:40.928287 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:40.928261 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56fbcc7474-jkckl"] Jul 10 17:54:41.716824 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.716787 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56fbcc7474-pd7cs"] Jul 10 17:54:41.717239 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.717184 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81bc6416-5ea7-498f-a85a-61eff3c9be9c" containerName="manager" Jul 10 17:54:41.717239 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.717206 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="81bc6416-5ea7-498f-a85a-61eff3c9be9c" containerName="manager" Jul 10 17:54:41.717316 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.717273 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="81bc6416-5ea7-498f-a85a-61eff3c9be9c" containerName="manager" Jul 10 17:54:41.721538 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.721515 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:41.725030 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.725006 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 17:54:41.725390 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.725016 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 10 17:54:41.725390 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.725022 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mj6ll\"" Jul 10 17:54:41.729020 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.729000 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56fbcc7474-pd7cs"] Jul 10 17:54:41.856395 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.856353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs5x4\" (UniqueName: \"kubernetes.io/projected/98424bb4-a006-41b7-9b54-6897d9813843-kube-api-access-zs5x4\") pod \"maas-controller-56fbcc7474-pd7cs\" (UID: \"98424bb4-a006-41b7-9b54-6897d9813843\") " pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:41.856581 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.856499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98424bb4-a006-41b7-9b54-6897d9813843-webhook-cert\") pod \"maas-controller-56fbcc7474-pd7cs\" (UID: \"98424bb4-a006-41b7-9b54-6897d9813843\") " pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:41.957358 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.957325 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs5x4\" (UniqueName: \"kubernetes.io/projected/98424bb4-a006-41b7-9b54-6897d9813843-kube-api-access-zs5x4\") pod \"maas-controller-56fbcc7474-pd7cs\" (UID: \"98424bb4-a006-41b7-9b54-6897d9813843\") " pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:41.957551 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.957412 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98424bb4-a006-41b7-9b54-6897d9813843-webhook-cert\") pod \"maas-controller-56fbcc7474-pd7cs\" (UID: \"98424bb4-a006-41b7-9b54-6897d9813843\") " pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:41.959904 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.959874 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98424bb4-a006-41b7-9b54-6897d9813843-webhook-cert\") pod \"maas-controller-56fbcc7474-pd7cs\" (UID: \"98424bb4-a006-41b7-9b54-6897d9813843\") " pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:41.965539 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:41.965511 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs5x4\" (UniqueName: \"kubernetes.io/projected/98424bb4-a006-41b7-9b54-6897d9813843-kube-api-access-zs5x4\") pod \"maas-controller-56fbcc7474-pd7cs\" (UID: \"98424bb4-a006-41b7-9b54-6897d9813843\") " pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:42.032741 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.032664 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:42.163103 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.163080 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56fbcc7474-pd7cs"] Jul 10 17:54:42.164954 ip-10-0-131-229 kubenswrapper[2568]: W0710 17:54:42.164927 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98424bb4_a006_41b7_9b54_6897d9813843.slice/crio-b8c4dc4d5ae4c068df53b26c5c700e5d9b5fb06c0fecfc206c689fd7d8adf215 WatchSource:0}: Error finding container b8c4dc4d5ae4c068df53b26c5c700e5d9b5fb06c0fecfc206c689fd7d8adf215: Status 404 returned error can't find the container with id b8c4dc4d5ae4c068df53b26c5c700e5d9b5fb06c0fecfc206c689fd7d8adf215 Jul 10 17:54:42.166355 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.166333 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 17:54:42.681512 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.681492 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 10 17:54:42.756979 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.756938 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81bc6416-5ea7-498f-a85a-61eff3c9be9c" path="/var/lib/kubelet/pods/81bc6416-5ea7-498f-a85a-61eff3c9be9c/volumes" Jul 10 17:54:42.924008 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.923956 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56fbcc7474-pd7cs" event={"ID":"98424bb4-a006-41b7-9b54-6897d9813843","Type":"ContainerStarted","Data":"e5d5cfeb92263e9a7123893d57869f8cd739e9d1d7af20ff6f819ea5faa3778b"} Jul 10 17:54:42.924144 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.924047 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56fbcc7474-pd7cs" event={"ID":"98424bb4-a006-41b7-9b54-6897d9813843","Type":"ContainerStarted","Data":"b8c4dc4d5ae4c068df53b26c5c700e5d9b5fb06c0fecfc206c689fd7d8adf215"} Jul 10 17:54:42.924144 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.924077 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:54:42.942029 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:42.941925 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56fbcc7474-pd7cs" podStartSLOduration=1.429251643 podStartE2EDuration="1.941910152s" totalImagesPullingTime="512.658509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:54:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:54:42.939602081 +0000 UTC m=+960.767250855" watchObservedRunningTime="2026-07-10 17:54:42.941910152 +0000 UTC m=+960.769558952" Jul 10 17:54:53.935946 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:54:53.935918 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56fbcc7474-pd7cs" Jul 10 17:58:46.353677 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:58:46.353651 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 17:58:46.356142 ip-10-0-131-229 kubenswrapper[2568]: I0710 17:58:46.355539 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:03:46.389251 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:03:46.389127 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:03:46.393187 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:03:46.392387 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:06:55.811755 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:55.811713 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc"] Jul 10 18:06:55.815631 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:55.815601 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:55.818540 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:55.818518 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-29a83520-serving-cert\"" Jul 10 18:06:55.828100 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:55.828080 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc"] Jul 10 18:06:55.912632 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:55.912589 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:55.912812 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:55.912650 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf6jt\" (UniqueName: \"kubernetes.io/projected/80f57a92-d27a-4fce-b812-e9cad2232b2f-kube-api-access-jf6jt\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:55.912812 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:55.912672 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f57a92-d27a-4fce-b812-e9cad2232b2f-openshift-service-ca\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.013875 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.013839 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.014070 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.013880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf6jt\" (UniqueName: \"kubernetes.io/projected/80f57a92-d27a-4fce-b812-e9cad2232b2f-kube-api-access-jf6jt\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.014070 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.013902 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f57a92-d27a-4fce-b812-e9cad2232b2f-openshift-service-ca\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.014070 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:06:56.014032 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-serving-cert: secret "maas-api-e2e-ait-29a83520-serving-cert" not found Jul 10 18:06:56.014196 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:06:56.014110 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls podName:80f57a92-d27a-4fce-b812-e9cad2232b2f nodeName:}" failed. No retries permitted until 2026-07-10 18:06:56.514087761 +0000 UTC m=+1694.341736526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls") pod "maas-api-e2e-ait-29a83520-878445b69-hc2pc" (UID: "80f57a92-d27a-4fce-b812-e9cad2232b2f") : secret "maas-api-e2e-ait-29a83520-serving-cert" not found Jul 10 18:06:56.014587 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.014568 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f57a92-d27a-4fce-b812-e9cad2232b2f-openshift-service-ca\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.028629 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.028605 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf6jt\" (UniqueName: \"kubernetes.io/projected/80f57a92-d27a-4fce-b812-e9cad2232b2f-kube-api-access-jf6jt\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.517771 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.517738 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.520387 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.520363 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls\") pod \"maas-api-e2e-ait-29a83520-878445b69-hc2pc\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.727334 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.727300 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:56.870323 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.870281 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc"] Jul 10 18:06:56.875802 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:06:56.875753 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f57a92_d27a_4fce_b812_e9cad2232b2f.slice/crio-eb0bb945242e0d9c93186895eb5ea5677d925119d3c649d3795a06da7b7d19de WatchSource:0}: Error finding container eb0bb945242e0d9c93186895eb5ea5677d925119d3c649d3795a06da7b7d19de: Status 404 returned error can't find the container with id eb0bb945242e0d9c93186895eb5ea5677d925119d3c649d3795a06da7b7d19de Jul 10 18:06:56.877418 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:56.877398 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:06:57.703531 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:57.703492 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" event={"ID":"80f57a92-d27a-4fce-b812-e9cad2232b2f","Type":"ContainerStarted","Data":"7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693"} Jul 10 18:06:57.703732 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:57.703553 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" event={"ID":"80f57a92-d27a-4fce-b812-e9cad2232b2f","Type":"ContainerStarted","Data":"eb0bb945242e0d9c93186895eb5ea5677d925119d3c649d3795a06da7b7d19de"} Jul 10 18:06:57.703732 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:57.703621 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:06:57.725228 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:06:57.725181 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" podStartSLOduration=2.326501033 podStartE2EDuration="2.725166046s" totalImagesPullingTime="398.665013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:06:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:06:57.722210832 +0000 UTC m=+1695.549859605" watchObservedRunningTime="2026-07-10 18:06:57.725166046 +0000 UTC m=+1695.552814837" Jul 10 18:07:01.174953 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.174923 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc"] Jul 10 18:07:01.175439 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.175212 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" podUID="80f57a92-d27a-4fce-b812-e9cad2232b2f" containerName="maas-api" containerID="cri-o://7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693" gracePeriod=30 Jul 10 18:07:01.427082 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.427020 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:07:01.463016 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.462956 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls\") pod \"80f57a92-d27a-4fce-b812-e9cad2232b2f\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " Jul 10 18:07:01.463190 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.463025 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f57a92-d27a-4fce-b812-e9cad2232b2f-openshift-service-ca\") pod \"80f57a92-d27a-4fce-b812-e9cad2232b2f\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " Jul 10 18:07:01.463190 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.463067 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf6jt\" (UniqueName: \"kubernetes.io/projected/80f57a92-d27a-4fce-b812-e9cad2232b2f-kube-api-access-jf6jt\") pod \"80f57a92-d27a-4fce-b812-e9cad2232b2f\" (UID: \"80f57a92-d27a-4fce-b812-e9cad2232b2f\") " Jul 10 18:07:01.463404 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.463375 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80f57a92-d27a-4fce-b812-e9cad2232b2f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80f57a92-d27a-4fce-b812-e9cad2232b2f" (UID: "80f57a92-d27a-4fce-b812-e9cad2232b2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:07:01.465442 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.465416 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80f57a92-d27a-4fce-b812-e9cad2232b2f" (UID: "80f57a92-d27a-4fce-b812-e9cad2232b2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:07:01.465553 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.465479 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80f57a92-d27a-4fce-b812-e9cad2232b2f-kube-api-access-jf6jt" (OuterVolumeSpecName: "kube-api-access-jf6jt") pod "80f57a92-d27a-4fce-b812-e9cad2232b2f" (UID: "80f57a92-d27a-4fce-b812-e9cad2232b2f"). InnerVolumeSpecName "kube-api-access-jf6jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:07:01.564635 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.564600 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf6jt\" (UniqueName: \"kubernetes.io/projected/80f57a92-d27a-4fce-b812-e9cad2232b2f-kube-api-access-jf6jt\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:01.564635 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.564629 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80f57a92-d27a-4fce-b812-e9cad2232b2f-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:01.564635 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.564640 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80f57a92-d27a-4fce-b812-e9cad2232b2f-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:01.727369 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.727271 2568 generic.go:356] "Generic (PLEG): container finished" podID="80f57a92-d27a-4fce-b812-e9cad2232b2f" containerID="7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693" exitCode=0 Jul 10 18:07:01.727369 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.727321 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" Jul 10 18:07:01.727369 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.727340 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" event={"ID":"80f57a92-d27a-4fce-b812-e9cad2232b2f","Type":"ContainerDied","Data":"7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693"} Jul 10 18:07:01.727581 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.727421 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc" event={"ID":"80f57a92-d27a-4fce-b812-e9cad2232b2f","Type":"ContainerDied","Data":"eb0bb945242e0d9c93186895eb5ea5677d925119d3c649d3795a06da7b7d19de"} Jul 10 18:07:01.727581 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.727451 2568 scope.go:117] "RemoveContainer" containerID="7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693" Jul 10 18:07:01.738312 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.738291 2568 scope.go:117] "RemoveContainer" containerID="7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693" Jul 10 18:07:01.738639 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:07:01.738618 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693\": container with ID starting with 7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693 not found: ID does not exist" containerID="7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693" Jul 10 18:07:01.738696 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.738651 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693"} err="failed to get container status \"7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693\": rpc error: code = NotFound desc = could not find container \"7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693\": container with ID starting with 7c4c47aeb06ce6b6bd7217c8b6ad1d9643fa9920d48fbc118b9127b3c0df5693 not found: ID does not exist" Jul 10 18:07:01.763686 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.763645 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc"] Jul 10 18:07:01.768617 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:01.768587 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-29a83520-878445b69-hc2pc"] Jul 10 18:07:02.759079 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:02.759039 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80f57a92-d27a-4fce-b812-e9cad2232b2f" path="/var/lib/kubelet/pods/80f57a92-d27a-4fce-b812-e9cad2232b2f/volumes" Jul 10 18:07:08.528133 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.528097 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg"] Jul 10 18:07:08.528486 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.528443 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80f57a92-d27a-4fce-b812-e9cad2232b2f" containerName="maas-api" Jul 10 18:07:08.528486 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.528454 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="80f57a92-d27a-4fce-b812-e9cad2232b2f" containerName="maas-api" Jul 10 18:07:08.528555 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.528526 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="80f57a92-d27a-4fce-b812-e9cad2232b2f" containerName="maas-api" Jul 10 18:07:08.532625 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.532608 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:08.536606 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.536585 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a73b4da0-serving-cert\"" Jul 10 18:07:08.557659 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.557636 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg"] Jul 10 18:07:08.629668 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.629634 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8rbg\" (UniqueName: \"kubernetes.io/projected/bed1c8bb-2988-4099-9a7a-ff82f0b21630-kube-api-access-w8rbg\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:08.629844 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.629676 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:08.629844 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.629802 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed1c8bb-2988-4099-9a7a-ff82f0b21630-openshift-service-ca\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:08.730672 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.730635 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed1c8bb-2988-4099-9a7a-ff82f0b21630-openshift-service-ca\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:08.730834 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.730689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8rbg\" (UniqueName: \"kubernetes.io/projected/bed1c8bb-2988-4099-9a7a-ff82f0b21630-kube-api-access-w8rbg\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:08.730834 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.730708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:08.730905 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:07:08.730843 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-serving-cert: secret "maas-api-e2e-ait-a73b4da0-serving-cert" not found Jul 10 18:07:08.730940 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:07:08.730914 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls podName:bed1c8bb-2988-4099-9a7a-ff82f0b21630 nodeName:}" failed. No retries permitted until 2026-07-10 18:07:09.23089933 +0000 UTC m=+1707.058548077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls") pod "maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" (UID: "bed1c8bb-2988-4099-9a7a-ff82f0b21630") : secret "maas-api-e2e-ait-a73b4da0-serving-cert" not found Jul 10 18:07:08.731392 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.731371 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed1c8bb-2988-4099-9a7a-ff82f0b21630-openshift-service-ca\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:08.757981 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:08.757936 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8rbg\" (UniqueName: \"kubernetes.io/projected/bed1c8bb-2988-4099-9a7a-ff82f0b21630-kube-api-access-w8rbg\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:09.234855 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:09.234816 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:09.237854 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:09.237818 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls\") pod \"maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:09.442720 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:09.442685 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:09.590206 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:09.590182 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg"] Jul 10 18:07:09.591499 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:07:09.591477 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbed1c8bb_2988_4099_9a7a_ff82f0b21630.slice/crio-7c064a9894a1a5517a0f0f3307dc1fa76f8adbe3d400c64c95953f93cad2cdb5 WatchSource:0}: Error finding container 7c064a9894a1a5517a0f0f3307dc1fa76f8adbe3d400c64c95953f93cad2cdb5: Status 404 returned error can't find the container with id 7c064a9894a1a5517a0f0f3307dc1fa76f8adbe3d400c64c95953f93cad2cdb5 Jul 10 18:07:09.774073 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:09.773990 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" event={"ID":"bed1c8bb-2988-4099-9a7a-ff82f0b21630","Type":"ContainerStarted","Data":"7c064a9894a1a5517a0f0f3307dc1fa76f8adbe3d400c64c95953f93cad2cdb5"} Jul 10 18:07:10.779708 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:10.779678 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" event={"ID":"bed1c8bb-2988-4099-9a7a-ff82f0b21630","Type":"ContainerStarted","Data":"8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc"} Jul 10 18:07:10.780098 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:10.779737 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:10.797873 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:10.797809 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" podStartSLOduration=2.439990922 podStartE2EDuration="2.797791071s" totalImagesPullingTime="357.800149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:07:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:07:10.797431649 +0000 UTC m=+1708.625080428" watchObservedRunningTime="2026-07-10 18:07:10.797791071 +0000 UTC m=+1708.625439842" Jul 10 18:07:13.827627 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:13.827593 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg"] Jul 10 18:07:13.828083 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:13.827875 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" podUID="bed1c8bb-2988-4099-9a7a-ff82f0b21630" containerName="maas-api" containerID="cri-o://8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc" gracePeriod=30 Jul 10 18:07:14.075213 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.075191 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:14.180381 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.180356 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8rbg\" (UniqueName: \"kubernetes.io/projected/bed1c8bb-2988-4099-9a7a-ff82f0b21630-kube-api-access-w8rbg\") pod \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " Jul 10 18:07:14.180563 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.180431 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls\") pod \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " Jul 10 18:07:14.180563 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.180479 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed1c8bb-2988-4099-9a7a-ff82f0b21630-openshift-service-ca\") pod \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\" (UID: \"bed1c8bb-2988-4099-9a7a-ff82f0b21630\") " Jul 10 18:07:14.180842 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.180818 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bed1c8bb-2988-4099-9a7a-ff82f0b21630-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bed1c8bb-2988-4099-9a7a-ff82f0b21630" (UID: "bed1c8bb-2988-4099-9a7a-ff82f0b21630"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:07:14.182554 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.182532 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bed1c8bb-2988-4099-9a7a-ff82f0b21630-kube-api-access-w8rbg" (OuterVolumeSpecName: "kube-api-access-w8rbg") pod "bed1c8bb-2988-4099-9a7a-ff82f0b21630" (UID: "bed1c8bb-2988-4099-9a7a-ff82f0b21630"). InnerVolumeSpecName "kube-api-access-w8rbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:07:14.182654 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.182590 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bed1c8bb-2988-4099-9a7a-ff82f0b21630" (UID: "bed1c8bb-2988-4099-9a7a-ff82f0b21630"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:07:14.281031 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.280997 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8rbg\" (UniqueName: \"kubernetes.io/projected/bed1c8bb-2988-4099-9a7a-ff82f0b21630-kube-api-access-w8rbg\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:14.281031 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.281022 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bed1c8bb-2988-4099-9a7a-ff82f0b21630-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:14.281031 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.281031 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bed1c8bb-2988-4099-9a7a-ff82f0b21630-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:14.800275 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.800245 2568 generic.go:356] "Generic (PLEG): container finished" podID="bed1c8bb-2988-4099-9a7a-ff82f0b21630" containerID="8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc" exitCode=0 Jul 10 18:07:14.800414 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.800288 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" Jul 10 18:07:14.800414 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.800290 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" event={"ID":"bed1c8bb-2988-4099-9a7a-ff82f0b21630","Type":"ContainerDied","Data":"8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc"} Jul 10 18:07:14.800509 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.800416 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg" event={"ID":"bed1c8bb-2988-4099-9a7a-ff82f0b21630","Type":"ContainerDied","Data":"7c064a9894a1a5517a0f0f3307dc1fa76f8adbe3d400c64c95953f93cad2cdb5"} Jul 10 18:07:14.800509 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.800442 2568 scope.go:117] "RemoveContainer" containerID="8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc" Jul 10 18:07:14.809464 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.809447 2568 scope.go:117] "RemoveContainer" containerID="8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc" Jul 10 18:07:14.809723 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:07:14.809706 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc\": container with ID starting with 8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc not found: ID does not exist" containerID="8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc" Jul 10 18:07:14.809770 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.809732 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc"} err="failed to get container status \"8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc\": rpc error: code = NotFound desc = could not find container \"8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc\": container with ID starting with 8a03b2d23e5b01458a4b696c45a250e0f84f3f4da1cef8dcedb55ca7791cf5dc not found: ID does not exist" Jul 10 18:07:14.819473 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.819450 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg"] Jul 10 18:07:14.823896 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:14.823874 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a73b4da0-b499cfcff-b7lxg"] Jul 10 18:07:16.756312 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:16.756271 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bed1c8bb-2988-4099-9a7a-ff82f0b21630" path="/var/lib/kubelet/pods/bed1c8bb-2988-4099-9a7a-ff82f0b21630/volumes" Jul 10 18:07:27.894932 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:27.894901 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2"] Jul 10 18:07:27.895350 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:27.895294 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bed1c8bb-2988-4099-9a7a-ff82f0b21630" containerName="maas-api" Jul 10 18:07:27.895350 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:27.895306 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bed1c8bb-2988-4099-9a7a-ff82f0b21630" containerName="maas-api" Jul 10 18:07:27.895426 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:27.895365 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bed1c8bb-2988-4099-9a7a-ff82f0b21630" containerName="maas-api" Jul 10 18:07:27.906174 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:27.906143 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:27.924893 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:27.924873 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7bc3b5cb-serving-cert\"" Jul 10 18:07:27.934577 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:27.934555 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2"] Jul 10 18:07:28.003739 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.003709 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6da85fff-7492-4cb2-9566-2fcc72455e34-openshift-service-ca\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.003917 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.003757 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6p5p\" (UniqueName: \"kubernetes.io/projected/6da85fff-7492-4cb2-9566-2fcc72455e34-kube-api-access-g6p5p\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.003917 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.003846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6da85fff-7492-4cb2-9566-2fcc72455e34-maas-api-tls\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.105361 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.105312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6da85fff-7492-4cb2-9566-2fcc72455e34-openshift-service-ca\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.105542 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.105373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6p5p\" (UniqueName: \"kubernetes.io/projected/6da85fff-7492-4cb2-9566-2fcc72455e34-kube-api-access-g6p5p\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.105542 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.105447 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6da85fff-7492-4cb2-9566-2fcc72455e34-maas-api-tls\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.106125 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.106100 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6da85fff-7492-4cb2-9566-2fcc72455e34-openshift-service-ca\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.108112 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.108092 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6da85fff-7492-4cb2-9566-2fcc72455e34-maas-api-tls\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.120048 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.120024 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6p5p\" (UniqueName: \"kubernetes.io/projected/6da85fff-7492-4cb2-9566-2fcc72455e34-kube-api-access-g6p5p\") pod \"maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.216101 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.216019 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:28.360555 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.360521 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2"] Jul 10 18:07:28.362009 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:07:28.361948 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da85fff_7492_4cb2_9566_2fcc72455e34.slice/crio-e51444e6dec503dbc1b52077f65adaea9dd50223feedd3cb447f3dbdf33aeab3 WatchSource:0}: Error finding container e51444e6dec503dbc1b52077f65adaea9dd50223feedd3cb447f3dbdf33aeab3: Status 404 returned error can't find the container with id e51444e6dec503dbc1b52077f65adaea9dd50223feedd3cb447f3dbdf33aeab3 Jul 10 18:07:28.879893 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:28.879863 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" event={"ID":"6da85fff-7492-4cb2-9566-2fcc72455e34","Type":"ContainerStarted","Data":"e51444e6dec503dbc1b52077f65adaea9dd50223feedd3cb447f3dbdf33aeab3"} Jul 10 18:07:29.887888 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:29.887853 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" event={"ID":"6da85fff-7492-4cb2-9566-2fcc72455e34","Type":"ContainerStarted","Data":"d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270"} Jul 10 18:07:29.888318 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:29.888006 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:33.021213 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.021164 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" podStartSLOduration=5.415057642 podStartE2EDuration="6.021148304s" totalImagesPullingTime="606.090662ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:07:29.906104068 +0000 UTC m=+1727.733752848" watchObservedRunningTime="2026-07-10 18:07:33.021148304 +0000 UTC m=+1730.848797072" Jul 10 18:07:33.022511 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.022492 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2"] Jul 10 18:07:33.022668 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.022649 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" podUID="6da85fff-7492-4cb2-9566-2fcc72455e34" containerName="maas-api" containerID="cri-o://d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270" gracePeriod=30 Jul 10 18:07:33.273580 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.273517 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:33.454687 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.454660 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6da85fff-7492-4cb2-9566-2fcc72455e34-maas-api-tls\") pod \"6da85fff-7492-4cb2-9566-2fcc72455e34\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " Jul 10 18:07:33.454886 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.454714 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6da85fff-7492-4cb2-9566-2fcc72455e34-openshift-service-ca\") pod \"6da85fff-7492-4cb2-9566-2fcc72455e34\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " Jul 10 18:07:33.454886 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.454775 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6p5p\" (UniqueName: \"kubernetes.io/projected/6da85fff-7492-4cb2-9566-2fcc72455e34-kube-api-access-g6p5p\") pod \"6da85fff-7492-4cb2-9566-2fcc72455e34\" (UID: \"6da85fff-7492-4cb2-9566-2fcc72455e34\") " Jul 10 18:07:33.455152 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.455127 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6da85fff-7492-4cb2-9566-2fcc72455e34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6da85fff-7492-4cb2-9566-2fcc72455e34" (UID: "6da85fff-7492-4cb2-9566-2fcc72455e34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:07:33.457004 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.456982 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6da85fff-7492-4cb2-9566-2fcc72455e34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6da85fff-7492-4cb2-9566-2fcc72455e34" (UID: "6da85fff-7492-4cb2-9566-2fcc72455e34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:07:33.457089 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.457064 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6da85fff-7492-4cb2-9566-2fcc72455e34-kube-api-access-g6p5p" (OuterVolumeSpecName: "kube-api-access-g6p5p") pod "6da85fff-7492-4cb2-9566-2fcc72455e34" (UID: "6da85fff-7492-4cb2-9566-2fcc72455e34"). InnerVolumeSpecName "kube-api-access-g6p5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:07:33.555708 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.555629 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6da85fff-7492-4cb2-9566-2fcc72455e34-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:33.555708 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.555656 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6da85fff-7492-4cb2-9566-2fcc72455e34-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:33.555708 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.555667 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6p5p\" (UniqueName: \"kubernetes.io/projected/6da85fff-7492-4cb2-9566-2fcc72455e34-kube-api-access-g6p5p\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:07:33.913224 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.913187 2568 generic.go:356] "Generic (PLEG): container finished" podID="6da85fff-7492-4cb2-9566-2fcc72455e34" containerID="d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270" exitCode=0 Jul 10 18:07:33.913391 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.913244 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" Jul 10 18:07:33.913391 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.913249 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" event={"ID":"6da85fff-7492-4cb2-9566-2fcc72455e34","Type":"ContainerDied","Data":"d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270"} Jul 10 18:07:33.913391 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.913315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2" event={"ID":"6da85fff-7492-4cb2-9566-2fcc72455e34","Type":"ContainerDied","Data":"e51444e6dec503dbc1b52077f65adaea9dd50223feedd3cb447f3dbdf33aeab3"} Jul 10 18:07:33.913391 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.913336 2568 scope.go:117] "RemoveContainer" containerID="d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270" Jul 10 18:07:33.922589 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.922571 2568 scope.go:117] "RemoveContainer" containerID="d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270" Jul 10 18:07:33.922854 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:07:33.922836 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270\": container with ID starting with d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270 not found: ID does not exist" containerID="d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270" Jul 10 18:07:33.922898 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.922862 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270"} err="failed to get container status \"d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270\": rpc error: code = NotFound desc = could not find container \"d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270\": container with ID starting with d9e90a2affb96f045f8bbc16ce718e989172a60d8b83f648e1bcebf8c9ee1270 not found: ID does not exist" Jul 10 18:07:33.945241 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.945209 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2"] Jul 10 18:07:33.953246 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:33.953220 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7bc3b5cb-5859c77f55-7mdg2"] Jul 10 18:07:34.756826 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:07:34.756749 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6da85fff-7492-4cb2-9566-2fcc72455e34" path="/var/lib/kubelet/pods/6da85fff-7492-4cb2-9566-2fcc72455e34/volumes" Jul 10 18:08:46.423436 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:08:46.423321 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:08:46.428420 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:08:46.426946 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:09:21.533939 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.533899 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5"] Jul 10 18:09:21.534586 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.534552 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6da85fff-7492-4cb2-9566-2fcc72455e34" containerName="maas-api" Jul 10 18:09:21.534725 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.534590 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6da85fff-7492-4cb2-9566-2fcc72455e34" containerName="maas-api" Jul 10 18:09:21.534786 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.534727 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6da85fff-7492-4cb2-9566-2fcc72455e34" containerName="maas-api" Jul 10 18:09:21.537950 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.537928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.540791 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.540768 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f6b351bc-serving-cert\"" Jul 10 18:09:21.554733 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.554712 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5"] Jul 10 18:09:21.655523 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.655492 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdpmf\" (UniqueName: \"kubernetes.io/projected/41b9afae-c5b4-4273-8411-e11aa857f5ec-kube-api-access-bdpmf\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.655677 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.655532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41b9afae-c5b4-4273-8411-e11aa857f5ec-openshift-service-ca\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.655677 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.655574 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41b9afae-c5b4-4273-8411-e11aa857f5ec-maas-api-tls\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.756254 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.756218 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdpmf\" (UniqueName: \"kubernetes.io/projected/41b9afae-c5b4-4273-8411-e11aa857f5ec-kube-api-access-bdpmf\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.756420 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.756262 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41b9afae-c5b4-4273-8411-e11aa857f5ec-openshift-service-ca\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.756420 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.756395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41b9afae-c5b4-4273-8411-e11aa857f5ec-maas-api-tls\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.756864 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.756840 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41b9afae-c5b4-4273-8411-e11aa857f5ec-openshift-service-ca\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.759101 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.759062 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41b9afae-c5b4-4273-8411-e11aa857f5ec-maas-api-tls\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.769730 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.769709 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdpmf\" (UniqueName: \"kubernetes.io/projected/41b9afae-c5b4-4273-8411-e11aa857f5ec-kube-api-access-bdpmf\") pod \"maas-api-e2e-mt-f6b351bc-677688b774-k97c5\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.847749 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.847686 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:21.993166 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:21.993128 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5"] Jul 10 18:09:21.994810 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:09:21.994783 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b9afae_c5b4_4273_8411_e11aa857f5ec.slice/crio-cc0884ce60e6d7d8d307cdee956b0a93d715a8601c1db7a4611468fd2500a271 WatchSource:0}: Error finding container cc0884ce60e6d7d8d307cdee956b0a93d715a8601c1db7a4611468fd2500a271: Status 404 returned error can't find the container with id cc0884ce60e6d7d8d307cdee956b0a93d715a8601c1db7a4611468fd2500a271 Jul 10 18:09:22.470408 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:22.470368 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" event={"ID":"41b9afae-c5b4-4273-8411-e11aa857f5ec","Type":"ContainerStarted","Data":"cc0884ce60e6d7d8d307cdee956b0a93d715a8601c1db7a4611468fd2500a271"} Jul 10 18:09:23.477320 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:23.477286 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" event={"ID":"41b9afae-c5b4-4273-8411-e11aa857f5ec","Type":"ContainerStarted","Data":"d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806"} Jul 10 18:09:23.477715 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:23.477359 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:23.495486 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:23.495436 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" podStartSLOduration=1.958683365 podStartE2EDuration="2.495419168s" totalImagesPullingTime="536.735803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:09:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:09:23.494235635 +0000 UTC m=+1841.321884403" watchObservedRunningTime="2026-07-10 18:09:23.495419168 +0000 UTC m=+1841.323067939" Jul 10 18:09:29.488568 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:29.488541 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:34.473139 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.473096 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5"] Jul 10 18:09:34.473667 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.473462 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" podUID="41b9afae-c5b4-4273-8411-e11aa857f5ec" containerName="maas-api" containerID="cri-o://d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806" gracePeriod=30 Jul 10 18:09:34.483940 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.483898 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" podUID="41b9afae-c5b4-4273-8411-e11aa857f5ec" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.48:8443/health\": dial tcp 10.132.0.48:8443: connect: connection refused" Jul 10 18:09:34.724481 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.724425 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:34.878258 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.878226 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41b9afae-c5b4-4273-8411-e11aa857f5ec-openshift-service-ca\") pod \"41b9afae-c5b4-4273-8411-e11aa857f5ec\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " Jul 10 18:09:34.878425 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.878286 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdpmf\" (UniqueName: \"kubernetes.io/projected/41b9afae-c5b4-4273-8411-e11aa857f5ec-kube-api-access-bdpmf\") pod \"41b9afae-c5b4-4273-8411-e11aa857f5ec\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " Jul 10 18:09:34.878425 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.878392 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41b9afae-c5b4-4273-8411-e11aa857f5ec-maas-api-tls\") pod \"41b9afae-c5b4-4273-8411-e11aa857f5ec\" (UID: \"41b9afae-c5b4-4273-8411-e11aa857f5ec\") " Jul 10 18:09:34.878661 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.878637 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41b9afae-c5b4-4273-8411-e11aa857f5ec-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "41b9afae-c5b4-4273-8411-e11aa857f5ec" (UID: "41b9afae-c5b4-4273-8411-e11aa857f5ec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:09:34.880540 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.880514 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41b9afae-c5b4-4273-8411-e11aa857f5ec-kube-api-access-bdpmf" (OuterVolumeSpecName: "kube-api-access-bdpmf") pod "41b9afae-c5b4-4273-8411-e11aa857f5ec" (UID: "41b9afae-c5b4-4273-8411-e11aa857f5ec"). InnerVolumeSpecName "kube-api-access-bdpmf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:09:34.880611 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.880595 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41b9afae-c5b4-4273-8411-e11aa857f5ec-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "41b9afae-c5b4-4273-8411-e11aa857f5ec" (UID: "41b9afae-c5b4-4273-8411-e11aa857f5ec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:09:34.980074 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.979957 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/41b9afae-c5b4-4273-8411-e11aa857f5ec-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:09:34.980074 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.980017 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/41b9afae-c5b4-4273-8411-e11aa857f5ec-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:09:34.980074 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:34.980031 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdpmf\" (UniqueName: \"kubernetes.io/projected/41b9afae-c5b4-4273-8411-e11aa857f5ec-kube-api-access-bdpmf\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:09:35.542290 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.542253 2568 generic.go:356] "Generic (PLEG): container finished" podID="41b9afae-c5b4-4273-8411-e11aa857f5ec" containerID="d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806" exitCode=0 Jul 10 18:09:35.542685 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.542300 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" Jul 10 18:09:35.542685 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.542342 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" event={"ID":"41b9afae-c5b4-4273-8411-e11aa857f5ec","Type":"ContainerDied","Data":"d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806"} Jul 10 18:09:35.542685 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.542401 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5" event={"ID":"41b9afae-c5b4-4273-8411-e11aa857f5ec","Type":"ContainerDied","Data":"cc0884ce60e6d7d8d307cdee956b0a93d715a8601c1db7a4611468fd2500a271"} Jul 10 18:09:35.542685 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.542432 2568 scope.go:117] "RemoveContainer" containerID="d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806" Jul 10 18:09:35.551834 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.551816 2568 scope.go:117] "RemoveContainer" containerID="d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806" Jul 10 18:09:35.552128 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:09:35.552110 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806\": container with ID starting with d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806 not found: ID does not exist" containerID="d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806" Jul 10 18:09:35.552211 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.552135 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806"} err="failed to get container status \"d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806\": rpc error: code = NotFound desc = could not find container \"d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806\": container with ID starting with d3a6ea74b66e9c6dbf7f363a21d49e4d1ae607a070ac3974cc48c6912eac1806 not found: ID does not exist" Jul 10 18:09:35.573453 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.573428 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5"] Jul 10 18:09:35.580318 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:35.580296 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f6b351bc-677688b774-k97c5"] Jul 10 18:09:36.756359 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:09:36.756328 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41b9afae-c5b4-4273-8411-e11aa857f5ec" path="/var/lib/kubelet/pods/41b9afae-c5b4-4273-8411-e11aa857f5ec/volumes" Jul 10 18:10:04.026897 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.026810 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b"] Jul 10 18:10:04.027365 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.027192 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b9afae-c5b4-4273-8411-e11aa857f5ec" containerName="maas-api" Jul 10 18:10:04.027365 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.027204 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b9afae-c5b4-4273-8411-e11aa857f5ec" containerName="maas-api" Jul 10 18:10:04.027365 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.027266 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="41b9afae-c5b4-4273-8411-e11aa857f5ec" containerName="maas-api" Jul 10 18:10:04.030434 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.030415 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.036470 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.036450 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ba6e3fc9-serving-cert\"" Jul 10 18:10:04.054706 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.054682 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b"] Jul 10 18:10:04.135844 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.135807 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2cd9259-920a-4453-b637-f4c36ac13382-maas-api-tls\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.135844 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.135841 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cd9259-920a-4453-b637-f4c36ac13382-openshift-service-ca\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.136097 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.135860 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx2qt\" (UniqueName: \"kubernetes.io/projected/a2cd9259-920a-4453-b637-f4c36ac13382-kube-api-access-nx2qt\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.236857 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.236828 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2cd9259-920a-4453-b637-f4c36ac13382-maas-api-tls\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.236857 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.236859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cd9259-920a-4453-b637-f4c36ac13382-openshift-service-ca\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.237104 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.236879 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx2qt\" (UniqueName: \"kubernetes.io/projected/a2cd9259-920a-4453-b637-f4c36ac13382-kube-api-access-nx2qt\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.237528 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.237504 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cd9259-920a-4453-b637-f4c36ac13382-openshift-service-ca\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.239421 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.239392 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2cd9259-920a-4453-b637-f4c36ac13382-maas-api-tls\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.248135 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.248116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx2qt\" (UniqueName: \"kubernetes.io/projected/a2cd9259-920a-4453-b637-f4c36ac13382-kube-api-access-nx2qt\") pod \"maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.340148 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.340070 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:04.485888 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.485860 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b"] Jul 10 18:10:04.488135 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:10:04.488110 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2cd9259_920a_4453_b637_f4c36ac13382.slice/crio-a8ae8a93ceb7463886a798d6b7e40c3b8bad9846d904f06ce717b8733088d1ee WatchSource:0}: Error finding container a8ae8a93ceb7463886a798d6b7e40c3b8bad9846d904f06ce717b8733088d1ee: Status 404 returned error can't find the container with id a8ae8a93ceb7463886a798d6b7e40c3b8bad9846d904f06ce717b8733088d1ee Jul 10 18:10:04.699585 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:04.699552 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" event={"ID":"a2cd9259-920a-4453-b637-f4c36ac13382","Type":"ContainerStarted","Data":"a8ae8a93ceb7463886a798d6b7e40c3b8bad9846d904f06ce717b8733088d1ee"} Jul 10 18:10:06.712765 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:06.712720 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" event={"ID":"a2cd9259-920a-4453-b637-f4c36ac13382","Type":"ContainerStarted","Data":"b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79"} Jul 10 18:10:06.713274 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:06.712878 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:06.732839 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:06.732782 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" podStartSLOduration=2.255626558 podStartE2EDuration="3.732767348s" totalImagesPullingTime="1.47714079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:10:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:10:06.731655896 +0000 UTC m=+1884.559304660" watchObservedRunningTime="2026-07-10 18:10:06.732767348 +0000 UTC m=+1884.560416116" Jul 10 18:10:12.723023 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:12.722995 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:41.278159 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.278127 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b"] Jul 10 18:10:41.278657 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.278332 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" podUID="a2cd9259-920a-4453-b637-f4c36ac13382" containerName="maas-api" containerID="cri-o://b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79" gracePeriod=30 Jul 10 18:10:41.520420 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.520397 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:41.589482 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.589398 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cd9259-920a-4453-b637-f4c36ac13382-openshift-service-ca\") pod \"a2cd9259-920a-4453-b637-f4c36ac13382\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " Jul 10 18:10:41.589482 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.589454 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2cd9259-920a-4453-b637-f4c36ac13382-maas-api-tls\") pod \"a2cd9259-920a-4453-b637-f4c36ac13382\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " Jul 10 18:10:41.589701 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.589511 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx2qt\" (UniqueName: \"kubernetes.io/projected/a2cd9259-920a-4453-b637-f4c36ac13382-kube-api-access-nx2qt\") pod \"a2cd9259-920a-4453-b637-f4c36ac13382\" (UID: \"a2cd9259-920a-4453-b637-f4c36ac13382\") " Jul 10 18:10:41.589758 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.589723 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2cd9259-920a-4453-b637-f4c36ac13382-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2cd9259-920a-4453-b637-f4c36ac13382" (UID: "a2cd9259-920a-4453-b637-f4c36ac13382"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:10:41.591549 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.591528 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2cd9259-920a-4453-b637-f4c36ac13382-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2cd9259-920a-4453-b637-f4c36ac13382" (UID: "a2cd9259-920a-4453-b637-f4c36ac13382"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:10:41.591660 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.591638 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2cd9259-920a-4453-b637-f4c36ac13382-kube-api-access-nx2qt" (OuterVolumeSpecName: "kube-api-access-nx2qt") pod "a2cd9259-920a-4453-b637-f4c36ac13382" (UID: "a2cd9259-920a-4453-b637-f4c36ac13382"). InnerVolumeSpecName "kube-api-access-nx2qt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:10:41.690189 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.690160 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx2qt\" (UniqueName: \"kubernetes.io/projected/a2cd9259-920a-4453-b637-f4c36ac13382-kube-api-access-nx2qt\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:10:41.690189 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.690186 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2cd9259-920a-4453-b637-f4c36ac13382-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:10:41.690189 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.690196 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2cd9259-920a-4453-b637-f4c36ac13382-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:10:41.896653 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.896624 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2cd9259-920a-4453-b637-f4c36ac13382" containerID="b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79" exitCode=0 Jul 10 18:10:41.896833 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.896669 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" Jul 10 18:10:41.896833 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.896701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" event={"ID":"a2cd9259-920a-4453-b637-f4c36ac13382","Type":"ContainerDied","Data":"b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79"} Jul 10 18:10:41.896833 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.896759 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b" event={"ID":"a2cd9259-920a-4453-b637-f4c36ac13382","Type":"ContainerDied","Data":"a8ae8a93ceb7463886a798d6b7e40c3b8bad9846d904f06ce717b8733088d1ee"} Jul 10 18:10:41.896833 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.896789 2568 scope.go:117] "RemoveContainer" containerID="b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79" Jul 10 18:10:41.906214 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.906194 2568 scope.go:117] "RemoveContainer" containerID="b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79" Jul 10 18:10:41.906497 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:10:41.906476 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79\": container with ID starting with b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79 not found: ID does not exist" containerID="b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79" Jul 10 18:10:41.906573 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.906510 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79"} err="failed to get container status \"b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79\": rpc error: code = NotFound desc = could not find container \"b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79\": container with ID starting with b15cced785837e5f99566f744a3ed65942031e1361c7bade25ed66cd4b3bad79 not found: ID does not exist" Jul 10 18:10:41.919198 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.919173 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b"] Jul 10 18:10:41.923368 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:41.923349 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba6e3fc9-6bfcd87f75-rt58b"] Jul 10 18:10:42.756265 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:10:42.756238 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2cd9259-920a-4453-b637-f4c36ac13382" path="/var/lib/kubelet/pods/a2cd9259-920a-4453-b637-f4c36ac13382/volumes" Jul 10 18:12:34.569120 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.569085 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg"] Jul 10 18:12:34.569716 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.569693 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2cd9259-920a-4453-b637-f4c36ac13382" containerName="maas-api" Jul 10 18:12:34.569802 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.569720 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2cd9259-920a-4453-b637-f4c36ac13382" containerName="maas-api" Jul 10 18:12:34.569869 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.569854 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2cd9259-920a-4453-b637-f4c36ac13382" containerName="maas-api" Jul 10 18:12:34.573117 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.573096 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:34.576832 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.576811 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-b1e501-serving-cert\"" Jul 10 18:12:34.593894 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.593871 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg"] Jul 10 18:12:34.666371 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.666340 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfkwp\" (UniqueName: \"kubernetes.io/projected/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-kube-api-access-wfkwp\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:34.666538 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.666382 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:34.666538 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.666433 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:34.766902 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.766871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfkwp\" (UniqueName: \"kubernetes.io/projected/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-kube-api-access-wfkwp\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:34.767093 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.766912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:34.767093 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.767029 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:34.767213 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:12:34.767178 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-serving-cert: secret "maas-api-e2e-inf-a-b1e501-serving-cert" not found Jul 10 18:12:34.767275 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:12:34.767263 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls podName:a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b nodeName:}" failed. No retries permitted until 2026-07-10 18:12:35.267242527 +0000 UTC m=+2033.094891277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls") pod "maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" (UID: "a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b") : secret "maas-api-e2e-inf-a-b1e501-serving-cert" not found Jul 10 18:12:34.767469 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.767450 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:34.778988 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:34.778937 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfkwp\" (UniqueName: \"kubernetes.io/projected/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-kube-api-access-wfkwp\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:35.271812 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:35.271782 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:35.274497 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:35.274473 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls\") pod \"maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:35.483308 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:35.483278 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:35.627150 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:35.627124 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg"] Jul 10 18:12:35.629960 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:12:35.629931 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27f32e9_6bd3_4c70_bfd2_17c80a1b0e1b.slice/crio-c4ecc1b878faed94bc4184ce9f51fb6367a4cf747208d566a9551991c33f4f61 WatchSource:0}: Error finding container c4ecc1b878faed94bc4184ce9f51fb6367a4cf747208d566a9551991c33f4f61: Status 404 returned error can't find the container with id c4ecc1b878faed94bc4184ce9f51fb6367a4cf747208d566a9551991c33f4f61 Jul 10 18:12:35.631380 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:35.631360 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 18:12:36.515045 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:36.514935 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" event={"ID":"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b","Type":"ContainerStarted","Data":"05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928"} Jul 10 18:12:36.515045 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:36.515025 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" event={"ID":"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b","Type":"ContainerStarted","Data":"c4ecc1b878faed94bc4184ce9f51fb6367a4cf747208d566a9551991c33f4f61"} Jul 10 18:12:36.515272 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:36.515139 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:36.534585 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:36.534537 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" podStartSLOduration=2.014835692 podStartE2EDuration="2.534522166s" totalImagesPullingTime="519.686474ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:12:36.532584554 +0000 UTC m=+2034.360233322" watchObservedRunningTime="2026-07-10 18:12:36.534522166 +0000 UTC m=+2034.362170981" Jul 10 18:12:42.504164 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.504135 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb"] Jul 10 18:12:42.507847 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.507825 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.510521 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.510498 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3fac88-serving-cert\"" Jul 10 18:12:42.525756 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.525735 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:12:42.529538 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.529512 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb"] Jul 10 18:12:42.642859 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.642828 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a88677-10ef-4864-8395-a9450bb1e9da-maas-api-tls\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.643071 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.642892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2hjs\" (UniqueName: \"kubernetes.io/projected/20a88677-10ef-4864-8395-a9450bb1e9da-kube-api-access-n2hjs\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.643071 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.642947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a88677-10ef-4864-8395-a9450bb1e9da-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.743467 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.743441 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a88677-10ef-4864-8395-a9450bb1e9da-maas-api-tls\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.743627 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.743496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2hjs\" (UniqueName: \"kubernetes.io/projected/20a88677-10ef-4864-8395-a9450bb1e9da-kube-api-access-n2hjs\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.743627 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.743536 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a88677-10ef-4864-8395-a9450bb1e9da-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.744156 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.744136 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a88677-10ef-4864-8395-a9450bb1e9da-openshift-service-ca\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.746085 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.746068 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-3fac88-serving-cert\"" Jul 10 18:12:42.754127 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.754103 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2hjs\" (UniqueName: \"kubernetes.io/projected/20a88677-10ef-4864-8395-a9450bb1e9da-kube-api-access-n2hjs\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.756493 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.756438 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a88677-10ef-4864-8395-a9450bb1e9da-maas-api-tls\") pod \"maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.818261 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.818239 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:42.960671 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:42.960650 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb"] Jul 10 18:12:42.962506 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:12:42.962479 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20a88677_10ef_4864_8395_a9450bb1e9da.slice/crio-45924268b37b31fc22a7c8d22ddbc5240a21281efe3edf472a0256bfc723b68c WatchSource:0}: Error finding container 45924268b37b31fc22a7c8d22ddbc5240a21281efe3edf472a0256bfc723b68c: Status 404 returned error can't find the container with id 45924268b37b31fc22a7c8d22ddbc5240a21281efe3edf472a0256bfc723b68c Jul 10 18:12:43.554004 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:43.553881 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" event={"ID":"20a88677-10ef-4864-8395-a9450bb1e9da","Type":"ContainerStarted","Data":"f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb"} Jul 10 18:12:43.554393 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:43.554014 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" event={"ID":"20a88677-10ef-4864-8395-a9450bb1e9da","Type":"ContainerStarted","Data":"45924268b37b31fc22a7c8d22ddbc5240a21281efe3edf472a0256bfc723b68c"} Jul 10 18:12:43.554393 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:43.554051 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:12:43.573928 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:43.573865 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" podStartSLOduration=1.237912264 podStartE2EDuration="1.573846547s" totalImagesPullingTime="335.934283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:12:43.57283799 +0000 UTC m=+2041.400486951" watchObservedRunningTime="2026-07-10 18:12:43.573846547 +0000 UTC m=+2041.401495316" Jul 10 18:12:49.564719 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:12:49.564688 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:13:46.459714 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:13:46.459602 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:13:46.464243 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:13:46.464230 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:14:03.079105 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.079054 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg"] Jul 10 18:14:03.079694 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.079668 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" podUID="a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" containerName="maas-api" containerID="cri-o://05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928" gracePeriod=30 Jul 10 18:14:03.335729 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.335666 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:14:03.471369 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.471337 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfkwp\" (UniqueName: \"kubernetes.io/projected/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-kube-api-access-wfkwp\") pod \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " Jul 10 18:14:03.471532 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.471404 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls\") pod \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " Jul 10 18:14:03.471532 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.471439 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-openshift-service-ca\") pod \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\" (UID: \"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b\") " Jul 10 18:14:03.471833 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.471810 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" (UID: "a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:14:03.473716 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.473690 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" (UID: "a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:14:03.473716 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.473705 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-kube-api-access-wfkwp" (OuterVolumeSpecName: "kube-api-access-wfkwp") pod "a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" (UID: "a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b"). InnerVolumeSpecName "kube-api-access-wfkwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:14:03.572905 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.572871 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfkwp\" (UniqueName: \"kubernetes.io/projected/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-kube-api-access-wfkwp\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:14:03.572905 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.572900 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:14:03.573124 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.572915 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:14:03.973409 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.973377 2568 generic.go:356] "Generic (PLEG): container finished" podID="a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" containerID="05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928" exitCode=0 Jul 10 18:14:03.973592 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.973424 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" Jul 10 18:14:03.973592 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.973472 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" event={"ID":"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b","Type":"ContainerDied","Data":"05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928"} Jul 10 18:14:03.973592 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.973528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg" event={"ID":"a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b","Type":"ContainerDied","Data":"c4ecc1b878faed94bc4184ce9f51fb6367a4cf747208d566a9551991c33f4f61"} Jul 10 18:14:03.973592 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.973554 2568 scope.go:117] "RemoveContainer" containerID="05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928" Jul 10 18:14:03.983745 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.983728 2568 scope.go:117] "RemoveContainer" containerID="05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928" Jul 10 18:14:03.984010 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:14:03.983989 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928\": container with ID starting with 05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928 not found: ID does not exist" containerID="05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928" Jul 10 18:14:03.984086 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:03.984017 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928"} err="failed to get container status \"05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928\": rpc error: code = NotFound desc = could not find container \"05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928\": container with ID starting with 05bba24098860ad8aa5dfbfe96061d1220ba4180f5344b59f1db51ecd8a16928 not found: ID does not exist" Jul 10 18:14:04.000622 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:04.000593 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg"] Jul 10 18:14:04.005566 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:04.005546 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b1e501-5c6c77f784-f6vdg"] Jul 10 18:14:04.756526 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:04.756485 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" path="/var/lib/kubelet/pods/a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b/volumes" Jul 10 18:14:46.784097 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:46.784064 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb"] Jul 10 18:14:46.784526 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:46.784313 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" podUID="20a88677-10ef-4864-8395-a9450bb1e9da" containerName="maas-api" containerID="cri-o://f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb" gracePeriod=30 Jul 10 18:14:47.043496 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.043432 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:14:47.163382 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.163346 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2hjs\" (UniqueName: \"kubernetes.io/projected/20a88677-10ef-4864-8395-a9450bb1e9da-kube-api-access-n2hjs\") pod \"20a88677-10ef-4864-8395-a9450bb1e9da\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " Jul 10 18:14:47.163575 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.163401 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a88677-10ef-4864-8395-a9450bb1e9da-openshift-service-ca\") pod \"20a88677-10ef-4864-8395-a9450bb1e9da\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " Jul 10 18:14:47.163575 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.163499 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a88677-10ef-4864-8395-a9450bb1e9da-maas-api-tls\") pod \"20a88677-10ef-4864-8395-a9450bb1e9da\" (UID: \"20a88677-10ef-4864-8395-a9450bb1e9da\") " Jul 10 18:14:47.163808 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.163782 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a88677-10ef-4864-8395-a9450bb1e9da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "20a88677-10ef-4864-8395-a9450bb1e9da" (UID: "20a88677-10ef-4864-8395-a9450bb1e9da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:14:47.165672 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.165644 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a88677-10ef-4864-8395-a9450bb1e9da-kube-api-access-n2hjs" (OuterVolumeSpecName: "kube-api-access-n2hjs") pod "20a88677-10ef-4864-8395-a9450bb1e9da" (UID: "20a88677-10ef-4864-8395-a9450bb1e9da"). InnerVolumeSpecName "kube-api-access-n2hjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:14:47.165774 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.165710 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20a88677-10ef-4864-8395-a9450bb1e9da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "20a88677-10ef-4864-8395-a9450bb1e9da" (UID: "20a88677-10ef-4864-8395-a9450bb1e9da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:14:47.197045 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.197012 2568 generic.go:356] "Generic (PLEG): container finished" podID="20a88677-10ef-4864-8395-a9450bb1e9da" containerID="f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb" exitCode=0 Jul 10 18:14:47.197189 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.197058 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" Jul 10 18:14:47.197189 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.197100 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" event={"ID":"20a88677-10ef-4864-8395-a9450bb1e9da","Type":"ContainerDied","Data":"f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb"} Jul 10 18:14:47.197189 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.197184 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb" event={"ID":"20a88677-10ef-4864-8395-a9450bb1e9da","Type":"ContainerDied","Data":"45924268b37b31fc22a7c8d22ddbc5240a21281efe3edf472a0256bfc723b68c"} Jul 10 18:14:47.197321 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.197208 2568 scope.go:117] "RemoveContainer" containerID="f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb" Jul 10 18:14:47.206342 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.206324 2568 scope.go:117] "RemoveContainer" containerID="f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb" Jul 10 18:14:47.206606 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:14:47.206586 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb\": container with ID starting with f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb not found: ID does not exist" containerID="f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb" Jul 10 18:14:47.206675 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.206619 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb"} err="failed to get container status \"f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb\": rpc error: code = NotFound desc = could not find container \"f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb\": container with ID starting with f7e4f4146733bb07a8e93666b9e0c1290dd625a5785f2934b82de918079c9bbb not found: ID does not exist" Jul 10 18:14:47.220749 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.220728 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb"] Jul 10 18:14:47.227485 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.227464 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-3fac88-554986fb7d-hzftb"] Jul 10 18:14:47.264946 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.264923 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2hjs\" (UniqueName: \"kubernetes.io/projected/20a88677-10ef-4864-8395-a9450bb1e9da-kube-api-access-n2hjs\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:14:47.264946 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.264944 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20a88677-10ef-4864-8395-a9450bb1e9da-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:14:47.265093 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:47.264956 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20a88677-10ef-4864-8395-a9450bb1e9da-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:14:48.756787 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:48.756756 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20a88677-10ef-4864-8395-a9450bb1e9da" path="/var/lib/kubelet/pods/20a88677-10ef-4864-8395-a9450bb1e9da/volumes" Jul 10 18:14:56.441933 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.441900 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5"] Jul 10 18:14:56.442343 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.442275 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" containerName="maas-api" Jul 10 18:14:56.442343 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.442286 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" containerName="maas-api" Jul 10 18:14:56.442343 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.442297 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20a88677-10ef-4864-8395-a9450bb1e9da" containerName="maas-api" Jul 10 18:14:56.442343 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.442302 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="20a88677-10ef-4864-8395-a9450bb1e9da" containerName="maas-api" Jul 10 18:14:56.442473 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.442386 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a27f32e9-6bd3-4c70-bfd2-17c80a1b0e1b" containerName="maas-api" Jul 10 18:14:56.442473 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.442398 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="20a88677-10ef-4864-8395-a9450bb1e9da" containerName="maas-api" Jul 10 18:14:56.446897 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.446858 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:56.449695 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.449669 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-dfa29f-serving-cert\"" Jul 10 18:14:56.542573 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.542543 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5"] Jul 10 18:14:56.546320 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.546297 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrx88\" (UniqueName: \"kubernetes.io/projected/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-kube-api-access-mrx88\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:56.546435 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.546346 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:56.546497 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.546474 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-openshift-service-ca\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:56.647190 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.647150 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:56.647368 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.647222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-openshift-service-ca\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:56.647368 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.647259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrx88\" (UniqueName: \"kubernetes.io/projected/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-kube-api-access-mrx88\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:56.647368 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:14:56.647312 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-serving-cert: secret "maas-api-e2e-api-a-dfa29f-serving-cert" not found Jul 10 18:14:56.647481 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:14:56.647386 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls podName:34a6ae4c-817e-4c4e-9e15-f5a40da71bfc nodeName:}" failed. No retries permitted until 2026-07-10 18:14:57.147369483 +0000 UTC m=+2174.975018232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls") pod "maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" (UID: "34a6ae4c-817e-4c4e-9e15-f5a40da71bfc") : secret "maas-api-e2e-api-a-dfa29f-serving-cert" not found Jul 10 18:14:56.647878 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.647858 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-openshift-service-ca\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:56.656929 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:56.656906 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrx88\" (UniqueName: \"kubernetes.io/projected/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-kube-api-access-mrx88\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:57.151911 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:57.151876 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:57.154553 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:57.154520 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls\") pod \"maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:57.358132 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:57.358095 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:57.502620 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:57.502588 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5"] Jul 10 18:14:57.503389 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:14:57.503354 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a6ae4c_817e_4c4e_9e15_f5a40da71bfc.slice/crio-660d000b66fc9b58a7121b52607e2211dc6822fe626cfb5968ad8c6b57679b87 WatchSource:0}: Error finding container 660d000b66fc9b58a7121b52607e2211dc6822fe626cfb5968ad8c6b57679b87: Status 404 returned error can't find the container with id 660d000b66fc9b58a7121b52607e2211dc6822fe626cfb5968ad8c6b57679b87 Jul 10 18:14:58.260506 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:58.260426 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" event={"ID":"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc","Type":"ContainerStarted","Data":"7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3"} Jul 10 18:14:58.260655 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:58.260506 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" event={"ID":"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc","Type":"ContainerStarted","Data":"660d000b66fc9b58a7121b52607e2211dc6822fe626cfb5968ad8c6b57679b87"} Jul 10 18:14:58.260655 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:58.260556 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:14:58.280673 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:14:58.280610 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" podStartSLOduration=1.7972949809999998 podStartE2EDuration="2.280592214s" totalImagesPullingTime="483.297233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:14:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:14:58.279193458 +0000 UTC m=+2176.106842227" watchObservedRunningTime="2026-07-10 18:14:58.280592214 +0000 UTC m=+2176.108240982" Jul 10 18:15:04.271570 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.271538 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:15:04.320053 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.320019 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5"] Jul 10 18:15:04.324257 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.324231 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:04.327885 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.327865 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ea3624-serving-cert\"" Jul 10 18:15:04.336649 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.336629 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5"] Jul 10 18:15:04.420855 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.420818 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pdst\" (UniqueName: \"kubernetes.io/projected/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-kube-api-access-8pdst\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:04.421062 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.420879 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-openshift-service-ca\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:04.421062 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.420996 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:04.522368 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.522271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pdst\" (UniqueName: \"kubernetes.io/projected/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-kube-api-access-8pdst\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:04.522368 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.522317 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-openshift-service-ca\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:04.522613 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.522393 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:04.522613 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:15:04.522502 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-serving-cert: secret "maas-api-e2e-api-b-ea3624-serving-cert" not found Jul 10 18:15:04.522613 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:15:04.522586 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls podName:faf301e2-d7ab-4a92-8c4e-d4d2121495a7 nodeName:}" failed. No retries permitted until 2026-07-10 18:15:05.022562071 +0000 UTC m=+2182.850210817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls") pod "maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" (UID: "faf301e2-d7ab-4a92-8c4e-d4d2121495a7") : secret "maas-api-e2e-api-b-ea3624-serving-cert" not found Jul 10 18:15:04.522935 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.522911 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-openshift-service-ca\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:04.535843 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:04.535818 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pdst\" (UniqueName: \"kubernetes.io/projected/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-kube-api-access-8pdst\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:05.028351 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:05.028320 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:05.031060 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:05.031036 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls\") pod \"maas-api-e2e-api-b-ea3624-595564b8fc-plcq5\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:05.236117 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:05.236086 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:05.409373 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:05.409350 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5"] Jul 10 18:15:05.411579 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:15:05.411554 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf301e2_d7ab_4a92_8c4e_d4d2121495a7.slice/crio-eee314e8ab462c75ee4cbba3b52dd6d4b73d62b4d0691279b00ef6302fb6a0a0 WatchSource:0}: Error finding container eee314e8ab462c75ee4cbba3b52dd6d4b73d62b4d0691279b00ef6302fb6a0a0: Status 404 returned error can't find the container with id eee314e8ab462c75ee4cbba3b52dd6d4b73d62b4d0691279b00ef6302fb6a0a0 Jul 10 18:15:06.304989 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:06.304929 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" event={"ID":"faf301e2-d7ab-4a92-8c4e-d4d2121495a7","Type":"ContainerStarted","Data":"75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa"} Jul 10 18:15:06.305195 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:06.305019 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" event={"ID":"faf301e2-d7ab-4a92-8c4e-d4d2121495a7","Type":"ContainerStarted","Data":"eee314e8ab462c75ee4cbba3b52dd6d4b73d62b4d0691279b00ef6302fb6a0a0"} Jul 10 18:15:06.305195 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:06.305141 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:06.335761 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:06.335710 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" podStartSLOduration=2.001281834 podStartE2EDuration="2.33569592s" totalImagesPullingTime="334.414086ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:15:06.334147615 +0000 UTC m=+2184.161796383" watchObservedRunningTime="2026-07-10 18:15:06.33569592 +0000 UTC m=+2184.163344688" Jul 10 18:15:12.315747 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:12.315715 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:17.616019 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.615984 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5"] Jul 10 18:15:17.616505 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.616238 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" podUID="34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" containerName="maas-api" containerID="cri-o://7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3" gracePeriod=30 Jul 10 18:15:17.869116 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.869049 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:15:17.942397 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.942365 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrx88\" (UniqueName: \"kubernetes.io/projected/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-kube-api-access-mrx88\") pod \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " Jul 10 18:15:17.942397 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.942399 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls\") pod \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " Jul 10 18:15:17.942584 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.942430 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-openshift-service-ca\") pod \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\" (UID: \"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc\") " Jul 10 18:15:17.942815 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.942784 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" (UID: "34a6ae4c-817e-4c4e-9e15-f5a40da71bfc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:15:17.944598 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.944575 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-kube-api-access-mrx88" (OuterVolumeSpecName: "kube-api-access-mrx88") pod "34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" (UID: "34a6ae4c-817e-4c4e-9e15-f5a40da71bfc"). InnerVolumeSpecName "kube-api-access-mrx88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:15:17.944679 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:17.944648 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" (UID: "34a6ae4c-817e-4c4e-9e15-f5a40da71bfc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:15:18.043436 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.043404 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrx88\" (UniqueName: \"kubernetes.io/projected/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-kube-api-access-mrx88\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:15:18.043436 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.043432 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:15:18.043436 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.043443 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:15:18.367560 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.367526 2568 generic.go:356] "Generic (PLEG): container finished" podID="34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" containerID="7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3" exitCode=0 Jul 10 18:15:18.367708 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.367574 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" Jul 10 18:15:18.367708 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.367612 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" event={"ID":"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc","Type":"ContainerDied","Data":"7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3"} Jul 10 18:15:18.367708 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.367665 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5" event={"ID":"34a6ae4c-817e-4c4e-9e15-f5a40da71bfc","Type":"ContainerDied","Data":"660d000b66fc9b58a7121b52607e2211dc6822fe626cfb5968ad8c6b57679b87"} Jul 10 18:15:18.367708 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.367687 2568 scope.go:117] "RemoveContainer" containerID="7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3" Jul 10 18:15:18.377649 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.377633 2568 scope.go:117] "RemoveContainer" containerID="7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3" Jul 10 18:15:18.377907 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:15:18.377888 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3\": container with ID starting with 7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3 not found: ID does not exist" containerID="7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3" Jul 10 18:15:18.377999 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.377918 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3"} err="failed to get container status \"7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3\": rpc error: code = NotFound desc = could not find container \"7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3\": container with ID starting with 7eed6b58ee822cd216e7ae89b5c68e48921d8590a1e94e03ccfd4c1defb0b8f3 not found: ID does not exist" Jul 10 18:15:18.390886 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.390862 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5"] Jul 10 18:15:18.395384 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.395366 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dfa29f-5bf89cfb6f-gnrk5"] Jul 10 18:15:18.756989 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:18.756875 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" path="/var/lib/kubelet/pods/34a6ae4c-817e-4c4e-9e15-f5a40da71bfc/volumes" Jul 10 18:15:24.862765 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:24.862727 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5"] Jul 10 18:15:24.863248 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:24.863188 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" podUID="faf301e2-d7ab-4a92-8c4e-d4d2121495a7" containerName="maas-api" containerID="cri-o://75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa" gracePeriod=30 Jul 10 18:15:25.129753 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.129730 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:25.203749 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.203719 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pdst\" (UniqueName: \"kubernetes.io/projected/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-kube-api-access-8pdst\") pod \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " Jul 10 18:15:25.203915 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.203796 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls\") pod \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " Jul 10 18:15:25.203915 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.203860 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-openshift-service-ca\") pod \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\" (UID: \"faf301e2-d7ab-4a92-8c4e-d4d2121495a7\") " Jul 10 18:15:25.204198 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.204171 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "faf301e2-d7ab-4a92-8c4e-d4d2121495a7" (UID: "faf301e2-d7ab-4a92-8c4e-d4d2121495a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:15:25.206028 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.206005 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-kube-api-access-8pdst" (OuterVolumeSpecName: "kube-api-access-8pdst") pod "faf301e2-d7ab-4a92-8c4e-d4d2121495a7" (UID: "faf301e2-d7ab-4a92-8c4e-d4d2121495a7"). InnerVolumeSpecName "kube-api-access-8pdst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:15:25.206141 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.206067 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "faf301e2-d7ab-4a92-8c4e-d4d2121495a7" (UID: "faf301e2-d7ab-4a92-8c4e-d4d2121495a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:15:25.305199 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.305140 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:15:25.305199 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.305196 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pdst\" (UniqueName: \"kubernetes.io/projected/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-kube-api-access-8pdst\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:15:25.305199 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.305210 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faf301e2-d7ab-4a92-8c4e-d4d2121495a7-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:15:25.413623 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.413531 2568 generic.go:356] "Generic (PLEG): container finished" podID="faf301e2-d7ab-4a92-8c4e-d4d2121495a7" containerID="75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa" exitCode=0 Jul 10 18:15:25.413623 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.413587 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" Jul 10 18:15:25.413623 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.413610 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" event={"ID":"faf301e2-d7ab-4a92-8c4e-d4d2121495a7","Type":"ContainerDied","Data":"75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa"} Jul 10 18:15:25.413840 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.413679 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5" event={"ID":"faf301e2-d7ab-4a92-8c4e-d4d2121495a7","Type":"ContainerDied","Data":"eee314e8ab462c75ee4cbba3b52dd6d4b73d62b4d0691279b00ef6302fb6a0a0"} Jul 10 18:15:25.413840 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.413701 2568 scope.go:117] "RemoveContainer" containerID="75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa" Jul 10 18:15:25.423002 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.422984 2568 scope.go:117] "RemoveContainer" containerID="75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa" Jul 10 18:15:25.423265 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:15:25.423245 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa\": container with ID starting with 75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa not found: ID does not exist" containerID="75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa" Jul 10 18:15:25.423358 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.423272 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa"} err="failed to get container status \"75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa\": rpc error: code = NotFound desc = could not find container \"75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa\": container with ID starting with 75317b768c451bea2814333c7a83d8afd55bc9bb8b0da14cad64ae79a7f7f3fa not found: ID does not exist" Jul 10 18:15:25.444139 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.444116 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5"] Jul 10 18:15:25.452286 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:25.452264 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ea3624-595564b8fc-plcq5"] Jul 10 18:15:26.756079 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:26.756051 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faf301e2-d7ab-4a92-8c4e-d4d2121495a7" path="/var/lib/kubelet/pods/faf301e2-d7ab-4a92-8c4e-d4d2121495a7/volumes" Jul 10 18:15:34.512616 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.512584 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd"] Jul 10 18:15:34.513000 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.512986 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" containerName="maas-api" Jul 10 18:15:34.513000 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.513001 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" containerName="maas-api" Jul 10 18:15:34.513082 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.513017 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf301e2-d7ab-4a92-8c4e-d4d2121495a7" containerName="maas-api" Jul 10 18:15:34.513082 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.513022 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf301e2-d7ab-4a92-8c4e-d4d2121495a7" containerName="maas-api" Jul 10 18:15:34.513154 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.513083 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="34a6ae4c-817e-4c4e-9e15-f5a40da71bfc" containerName="maas-api" Jul 10 18:15:34.513154 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.513094 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="faf301e2-d7ab-4a92-8c4e-d4d2121495a7" containerName="maas-api" Jul 10 18:15:34.517400 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.517379 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:34.519835 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.519814 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b8bbaf-serving-cert\"" Jul 10 18:15:34.529547 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.529525 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd"] Jul 10 18:15:34.592845 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.592815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a253967-cb83-4c00-b2a8-445c960e4d92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:34.592845 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.592850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6wdz\" (UniqueName: \"kubernetes.io/projected/6a253967-cb83-4c00-b2a8-445c960e4d92-kube-api-access-b6wdz\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:34.593117 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.592910 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:34.694373 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.694339 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a253967-cb83-4c00-b2a8-445c960e4d92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:34.694568 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.694382 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6wdz\" (UniqueName: \"kubernetes.io/projected/6a253967-cb83-4c00-b2a8-445c960e4d92-kube-api-access-b6wdz\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:34.694568 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.694451 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:34.694688 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:15:34.694591 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-serving-cert: secret "maas-api-e2e-shared-a-b8bbaf-serving-cert" not found Jul 10 18:15:34.694688 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:15:34.694679 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls podName:6a253967-cb83-4c00-b2a8-445c960e4d92 nodeName:}" failed. No retries permitted until 2026-07-10 18:15:35.194655974 +0000 UTC m=+2213.022304737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls") pod "maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" (UID: "6a253967-cb83-4c00-b2a8-445c960e4d92") : secret "maas-api-e2e-shared-a-b8bbaf-serving-cert" not found Jul 10 18:15:34.695084 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.695064 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a253967-cb83-4c00-b2a8-445c960e4d92-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:34.708512 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:34.708484 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6wdz\" (UniqueName: \"kubernetes.io/projected/6a253967-cb83-4c00-b2a8-445c960e4d92-kube-api-access-b6wdz\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:35.199901 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:35.199871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:35.202558 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:35.202534 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls\") pod \"maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:35.428684 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:35.428641 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:35.571614 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:35.571588 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd"] Jul 10 18:15:35.573561 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:15:35.573531 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a253967_cb83_4c00_b2a8_445c960e4d92.slice/crio-72de19ed0147b8f73158d8db9820b0b52265a0ee59ff8c6e04887d7516a30284 WatchSource:0}: Error finding container 72de19ed0147b8f73158d8db9820b0b52265a0ee59ff8c6e04887d7516a30284: Status 404 returned error can't find the container with id 72de19ed0147b8f73158d8db9820b0b52265a0ee59ff8c6e04887d7516a30284 Jul 10 18:15:36.474606 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:36.474564 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" event={"ID":"6a253967-cb83-4c00-b2a8-445c960e4d92","Type":"ContainerStarted","Data":"9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9"} Jul 10 18:15:36.474827 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:36.474643 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:36.474827 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:36.474663 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" event={"ID":"6a253967-cb83-4c00-b2a8-445c960e4d92","Type":"ContainerStarted","Data":"72de19ed0147b8f73158d8db9820b0b52265a0ee59ff8c6e04887d7516a30284"} Jul 10 18:15:36.495212 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:36.495168 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" podStartSLOduration=2.156120381 podStartE2EDuration="2.495152478s" totalImagesPullingTime="339.032097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:15:36.493628309 +0000 UTC m=+2214.321277079" watchObservedRunningTime="2026-07-10 18:15:36.495152478 +0000 UTC m=+2214.322801246" Jul 10 18:15:42.440785 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.440742 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l"] Jul 10 18:15:42.444522 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.444501 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.446921 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.446894 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-0af48d-serving-cert\"" Jul 10 18:15:42.457056 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.456992 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l"] Jul 10 18:15:42.486699 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.486673 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:15:42.569844 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.569806 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5znj\" (UniqueName: \"kubernetes.io/projected/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-kube-api-access-r5znj\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.570018 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.569854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.570071 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.570047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.670931 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.670897 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.671119 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.670952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.671119 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.671020 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5znj\" (UniqueName: \"kubernetes.io/projected/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-kube-api-access-r5znj\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.671585 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.671563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.673434 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.673416 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-0af48d-serving-cert\"" Jul 10 18:15:42.679337 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:42.679315 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5znj\" (UniqueName: \"kubernetes.io/projected/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-kube-api-access-r5znj\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:42.682085 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:15:42.682065 2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-serving-cert: secret "maas-api-e2e-shared-b-0af48d-serving-cert" not found Jul 10 18:15:42.682198 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:15:42.682116 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls podName:3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0 nodeName:}" failed. No retries permitted until 2026-07-10 18:15:43.182102095 +0000 UTC m=+2221.009750841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls") pod "maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" (UID: "3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0") : secret "maas-api-e2e-shared-b-0af48d-serving-cert" not found Jul 10 18:15:43.276444 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:43.276358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:43.278885 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:43.278859 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:43.358372 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:43.358341 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:43.507044 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:43.507017 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l"] Jul 10 18:15:43.508743 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:15:43.508713 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef3cf7a_cc0e_46d9_8665_19a8d6baa6e0.slice/crio-4f7f4c4674478027daff83d3efda898d2a70f24924b57989edc1b19d4633eb9b WatchSource:0}: Error finding container 4f7f4c4674478027daff83d3efda898d2a70f24924b57989edc1b19d4633eb9b: Status 404 returned error can't find the container with id 4f7f4c4674478027daff83d3efda898d2a70f24924b57989edc1b19d4633eb9b Jul 10 18:15:44.519436 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:44.519403 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" event={"ID":"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0","Type":"ContainerStarted","Data":"270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588"} Jul 10 18:15:44.519869 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:44.519456 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" event={"ID":"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0","Type":"ContainerStarted","Data":"4f7f4c4674478027daff83d3efda898d2a70f24924b57989edc1b19d4633eb9b"} Jul 10 18:15:44.519869 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:44.519555 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:15:44.539010 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:44.538943 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" podStartSLOduration=2.091028755 podStartE2EDuration="2.538929618s" totalImagesPullingTime="447.900863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 18:15:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:15:44.536586426 +0000 UTC m=+2222.364235194" watchObservedRunningTime="2026-07-10 18:15:44.538929618 +0000 UTC m=+2222.366578421" Jul 10 18:15:50.531550 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:15:50.531514 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:18:46.495016 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:18:46.494901 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:18:46.500101 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:18:46.500085 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7hnvb_5fc85324-2152-4dfd-8ace-99be0e85b860/ovn-acl-logging/0.log" Jul 10 18:22:29.752908 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:29.752820 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd"] Jul 10 18:22:29.753492 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:29.753076 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" podUID="6a253967-cb83-4c00-b2a8-445c960e4d92" containerName="maas-api" containerID="cri-o://9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9" gracePeriod=30 Jul 10 18:22:30.008309 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.008250 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:22:30.159820 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.159790 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6wdz\" (UniqueName: \"kubernetes.io/projected/6a253967-cb83-4c00-b2a8-445c960e4d92-kube-api-access-b6wdz\") pod \"6a253967-cb83-4c00-b2a8-445c960e4d92\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " Jul 10 18:22:30.160018 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.159843 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls\") pod \"6a253967-cb83-4c00-b2a8-445c960e4d92\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " Jul 10 18:22:30.160018 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.159873 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a253967-cb83-4c00-b2a8-445c960e4d92-openshift-service-ca\") pod \"6a253967-cb83-4c00-b2a8-445c960e4d92\" (UID: \"6a253967-cb83-4c00-b2a8-445c960e4d92\") " Jul 10 18:22:30.160311 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.160279 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a253967-cb83-4c00-b2a8-445c960e4d92-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6a253967-cb83-4c00-b2a8-445c960e4d92" (UID: "6a253967-cb83-4c00-b2a8-445c960e4d92"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:22:30.162370 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.162338 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a253967-cb83-4c00-b2a8-445c960e4d92-kube-api-access-b6wdz" (OuterVolumeSpecName: "kube-api-access-b6wdz") pod "6a253967-cb83-4c00-b2a8-445c960e4d92" (UID: "6a253967-cb83-4c00-b2a8-445c960e4d92"). InnerVolumeSpecName "kube-api-access-b6wdz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:22:30.162370 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.162358 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6a253967-cb83-4c00-b2a8-445c960e4d92" (UID: "6a253967-cb83-4c00-b2a8-445c960e4d92"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:22:30.261009 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.260905 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a253967-cb83-4c00-b2a8-445c960e4d92-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:22:30.261009 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.260933 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a253967-cb83-4c00-b2a8-445c960e4d92-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:22:30.261009 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.260944 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6wdz\" (UniqueName: \"kubernetes.io/projected/6a253967-cb83-4c00-b2a8-445c960e4d92-kube-api-access-b6wdz\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:22:30.678646 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.678608 2568 generic.go:356] "Generic (PLEG): container finished" podID="6a253967-cb83-4c00-b2a8-445c960e4d92" containerID="9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9" exitCode=0 Jul 10 18:22:30.678824 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.678654 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" Jul 10 18:22:30.678824 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.678707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" event={"ID":"6a253967-cb83-4c00-b2a8-445c960e4d92","Type":"ContainerDied","Data":"9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9"} Jul 10 18:22:30.678824 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.678762 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd" event={"ID":"6a253967-cb83-4c00-b2a8-445c960e4d92","Type":"ContainerDied","Data":"72de19ed0147b8f73158d8db9820b0b52265a0ee59ff8c6e04887d7516a30284"} Jul 10 18:22:30.678824 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.678783 2568 scope.go:117] "RemoveContainer" containerID="9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9" Jul 10 18:22:30.689121 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.689101 2568 scope.go:117] "RemoveContainer" containerID="9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9" Jul 10 18:22:30.689396 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:22:30.689380 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9\": container with ID starting with 9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9 not found: ID does not exist" containerID="9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9" Jul 10 18:22:30.689461 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.689405 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9"} err="failed to get container status \"9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9\": rpc error: code = NotFound desc = could not find container \"9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9\": container with ID starting with 9f3cfbc20459a16694d4b19d279f189e46ccaf64a6a7ad781e8d06b7a09249e9 not found: ID does not exist" Jul 10 18:22:30.701096 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.701068 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd"] Jul 10 18:22:30.703478 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.703451 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b8bbaf-79d8bbb9d9-fzfjd"] Jul 10 18:22:30.756077 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:30.756051 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a253967-cb83-4c00-b2a8-445c960e4d92" path="/var/lib/kubelet/pods/6a253967-cb83-4c00-b2a8-445c960e4d92/volumes" Jul 10 18:22:36.827505 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:36.827467 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l"] Jul 10 18:22:36.827940 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:36.827792 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" podUID="3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" containerName="maas-api" containerID="cri-o://270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588" gracePeriod=30 Jul 10 18:22:37.281298 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.281273 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:22:37.424438 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.424337 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-openshift-service-ca\") pod \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " Jul 10 18:22:37.424438 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.424396 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls\") pod \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " Jul 10 18:22:37.424438 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.424431 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5znj\" (UniqueName: \"kubernetes.io/projected/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-kube-api-access-r5znj\") pod \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\" (UID: \"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0\") " Jul 10 18:22:37.424786 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.424755 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" (UID: "3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 18:22:37.426716 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.426689 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-kube-api-access-r5znj" (OuterVolumeSpecName: "kube-api-access-r5znj") pod "3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" (UID: "3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0"). InnerVolumeSpecName "kube-api-access-r5znj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 18:22:37.426834 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.426787 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" (UID: "3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 18:22:37.525495 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.525456 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-maas-api-tls\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:22:37.525495 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.525492 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5znj\" (UniqueName: \"kubernetes.io/projected/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-kube-api-access-r5znj\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:22:37.525495 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.525502 2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0-openshift-service-ca\") on node \"ip-10-0-131-229.ec2.internal\" DevicePath \"\"" Jul 10 18:22:37.718704 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.718617 2568 generic.go:356] "Generic (PLEG): container finished" podID="3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" containerID="270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588" exitCode=0 Jul 10 18:22:37.718704 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.718664 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" Jul 10 18:22:37.718876 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.718705 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" event={"ID":"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0","Type":"ContainerDied","Data":"270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588"} Jul 10 18:22:37.718876 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.718756 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l" event={"ID":"3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0","Type":"ContainerDied","Data":"4f7f4c4674478027daff83d3efda898d2a70f24924b57989edc1b19d4633eb9b"} Jul 10 18:22:37.718876 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.718779 2568 scope.go:117] "RemoveContainer" containerID="270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588" Jul 10 18:22:37.731379 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.731089 2568 scope.go:117] "RemoveContainer" containerID="270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588" Jul 10 18:22:37.731886 ip-10-0-131-229 kubenswrapper[2568]: E0710 18:22:37.731857 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588\": container with ID starting with 270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588 not found: ID does not exist" containerID="270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588" Jul 10 18:22:37.732057 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.731896 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588"} err="failed to get container status \"270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588\": rpc error: code = NotFound desc = could not find container \"270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588\": container with ID starting with 270bf344553ea19318879902bbe5846eb8bbb46b6f854cadef65ba9b55ba5588 not found: ID does not exist" Jul 10 18:22:37.742178 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.742156 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l"] Jul 10 18:22:37.751632 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:37.746272 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-0af48d-78499496fb-mhr5l"] Jul 10 18:22:38.755936 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:38.755895 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" path="/var/lib/kubelet/pods/3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0/volumes" Jul 10 18:22:48.212397 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:48.212351 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56fbcc7474-pd7cs_98424bb4-a006-41b7-9b54-6897d9813843/manager/0.log" Jul 10 18:22:48.672805 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:48.672778 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfdbc665d-tk6xb_cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d/copy-manifests/0.log" Jul 10 18:22:48.684660 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:48.684636 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfdbc665d-tk6xb_cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d/manager/0.log" Jul 10 18:22:49.163478 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:49.163441 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-b96984f7d-kthtc_a4a128c9-acd8-4c93-a3bd-624cd318de0f/maas-api/0.log" Jul 10 18:22:49.270034 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:49.270005 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-782ns_1f5c97eb-155a-4bdc-9dca-7506cb6e0d6a/postgres/0.log" Jul 10 18:22:50.010669 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.010638 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8_6ccaa546-0100-415f-95bb-07d3b2f0391a/extract/0.log" Jul 10 18:22:50.016724 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.016698 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8_6ccaa546-0100-415f-95bb-07d3b2f0391a/util/0.log" Jul 10 18:22:50.022478 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.022461 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8_6ccaa546-0100-415f-95bb-07d3b2f0391a/pull/0.log" Jul 10 18:22:50.131552 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.131521 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk_c782e8c3-54ee-4785-9876-71162727063b/util/0.log" Jul 10 18:22:50.137429 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.137402 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk_c782e8c3-54ee-4785-9876-71162727063b/pull/0.log" Jul 10 18:22:50.143190 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.143171 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk_c782e8c3-54ee-4785-9876-71162727063b/extract/0.log" Jul 10 18:22:50.262700 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.262607 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx_de8c0e60-699f-45bd-86be-24e61e9cb454/util/0.log" Jul 10 18:22:50.269812 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.269790 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx_de8c0e60-699f-45bd-86be-24e61e9cb454/pull/0.log" Jul 10 18:22:50.275675 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.275638 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx_de8c0e60-699f-45bd-86be-24e61e9cb454/extract/0.log" Jul 10 18:22:50.388776 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.388748 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf_120ec37c-c555-4ae9-b6c1-5af55f2e33f6/util/0.log" Jul 10 18:22:50.394323 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.394296 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf_120ec37c-c555-4ae9-b6c1-5af55f2e33f6/pull/0.log" Jul 10 18:22:50.399919 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:50.399902 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf_120ec37c-c555-4ae9-b6c1-5af55f2e33f6/extract/0.log" Jul 10 18:22:51.791241 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:51.791197 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tr8vw_ec0a9fc2-7081-44c2-b255-b7f49203c594/discovery/0.log" Jul 10 18:22:52.358303 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:52.358259 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bzllf_9b099279-a74f-4ed1-b641-e5d60c4d037e/payload-processing/0.log" Jul 10 18:22:52.470581 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:52.470556 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77d769cf84-rmlf4_8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c/router/0.log" Jul 10 18:22:52.815077 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:52.815005 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lnz7t_d915f4f9-296c-408a-ba07-a982a7285962/main/0.log" Jul 10 18:22:53.034150 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:53.034119 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hw6t5_9a64dcf5-fb52-4eac-8f2f-c6c08e6f8f19/main/0.log" Jul 10 18:22:59.965178 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:22:59.965145 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9q7pt_329c89c4-cfec-4be0-a5a8-aae54e2381b6/global-pull-secret-syncer/0.log" Jul 10 18:23:00.130455 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:00.130426 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rkvwl_bcd26b75-8c01-4a03-ba75-ba63b39fe0bb/konnectivity-agent/0.log" Jul 10 18:23:00.227314 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:00.227211 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-229.ec2.internal_fe6cda58f405aaf9348408762755bc73/haproxy/0.log" Jul 10 18:23:04.296301 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.296273 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8_6ccaa546-0100-415f-95bb-07d3b2f0391a/extract/0.log" Jul 10 18:23:04.318092 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.318056 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8_6ccaa546-0100-415f-95bb-07d3b2f0391a/util/0.log" Jul 10 18:23:04.338993 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.338962 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592f7v8_6ccaa546-0100-415f-95bb-07d3b2f0391a/pull/0.log" Jul 10 18:23:04.387081 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.387060 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk_c782e8c3-54ee-4785-9876-71162727063b/extract/0.log" Jul 10 18:23:04.409983 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.409948 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk_c782e8c3-54ee-4785-9876-71162727063b/util/0.log" Jul 10 18:23:04.428244 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.428224 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dnfmk_c782e8c3-54ee-4785-9876-71162727063b/pull/0.log" Jul 10 18:23:04.465257 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.465239 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx_de8c0e60-699f-45bd-86be-24e61e9cb454/extract/0.log" Jul 10 18:23:04.490518 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.490496 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx_de8c0e60-699f-45bd-86be-24e61e9cb454/util/0.log" Jul 10 18:23:04.511408 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.511383 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73g9hhx_de8c0e60-699f-45bd-86be-24e61e9cb454/pull/0.log" Jul 10 18:23:04.546186 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.546168 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf_120ec37c-c555-4ae9-b6c1-5af55f2e33f6/extract/0.log" Jul 10 18:23:04.570594 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.570523 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf_120ec37c-c555-4ae9-b6c1-5af55f2e33f6/util/0.log" Jul 10 18:23:04.600457 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:04.600439 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef19zbzf_120ec37c-c555-4ae9-b6c1-5af55f2e33f6/pull/0.log" Jul 10 18:23:07.576587 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:07.576561 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5968457d4f-hz4qg_bb488eba-86ea-4d04-a901-597cd1edce8f/metrics-server/0.log" Jul 10 18:23:07.651150 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:07.651125 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6p5c_946fb184-bf08-4f31-8ae3-37e1d2891cba/node-exporter/0.log" Jul 10 18:23:07.677473 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:07.677399 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6p5c_946fb184-bf08-4f31-8ae3-37e1d2891cba/kube-rbac-proxy/0.log" Jul 10 18:23:07.698865 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:07.698846 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6p5c_946fb184-bf08-4f31-8ae3-37e1d2891cba/init-textfile/0.log" Jul 10 18:23:08.143218 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.143190 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-9http_3dfe8a3d-13a6-430b-9cb7-6000c7b00827/prometheus-operator/0.log" Jul 10 18:23:08.165477 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.165451 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-9http_3dfe8a3d-13a6-430b-9cb7-6000c7b00827/kube-rbac-proxy/0.log" Jul 10 18:23:08.190635 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.190614 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-czkw6_565eecbc-a422-4a3c-9e14-9b660e2a938d/prometheus-operator-admission-webhook/0.log" Jul 10 18:23:08.220377 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.220356 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7698767496-lf5z5_7dc2c7b4-ec4e-461f-9008-a574c38f458a/telemeter-client/0.log" Jul 10 18:23:08.240151 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.240127 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7698767496-lf5z5_7dc2c7b4-ec4e-461f-9008-a574c38f458a/reload/0.log" Jul 10 18:23:08.269265 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.269248 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7698767496-lf5z5_7dc2c7b4-ec4e-461f-9008-a574c38f458a/kube-rbac-proxy/0.log" Jul 10 18:23:08.306516 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.306497 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655b67cf94-vb9rq_10d50cd8-583f-4b31-9667-4079d4a71a72/thanos-query/0.log" Jul 10 18:23:08.328421 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.328402 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655b67cf94-vb9rq_10d50cd8-583f-4b31-9667-4079d4a71a72/kube-rbac-proxy-web/0.log" Jul 10 18:23:08.351836 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.351815 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655b67cf94-vb9rq_10d50cd8-583f-4b31-9667-4079d4a71a72/kube-rbac-proxy/0.log" Jul 10 18:23:08.372314 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.372294 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655b67cf94-vb9rq_10d50cd8-583f-4b31-9667-4079d4a71a72/prom-label-proxy/0.log" Jul 10 18:23:08.393021 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.392959 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655b67cf94-vb9rq_10d50cd8-583f-4b31-9667-4079d4a71a72/kube-rbac-proxy-rules/0.log" Jul 10 18:23:08.417310 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:08.417240 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-655b67cf94-vb9rq_10d50cd8-583f-4b31-9667-4079d4a71a72/kube-rbac-proxy-metrics/0.log" Jul 10 18:23:09.588128 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:09.588089 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-gl62k_b32a2e4f-6d0b-4933-b32c-3d14bcd50eee/networking-console-plugin/0.log" Jul 10 18:23:10.544957 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.544926 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf"] Jul 10 18:23:10.545396 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.545378 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a253967-cb83-4c00-b2a8-445c960e4d92" containerName="maas-api" Jul 10 18:23:10.545502 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.545399 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a253967-cb83-4c00-b2a8-445c960e4d92" containerName="maas-api" Jul 10 18:23:10.545502 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.545421 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" containerName="maas-api" Jul 10 18:23:10.545502 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.545429 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" containerName="maas-api" Jul 10 18:23:10.545654 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.545535 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a253967-cb83-4c00-b2a8-445c960e4d92" containerName="maas-api" Jul 10 18:23:10.545654 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.545549 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ef3cf7a-cc0e-46d9-8665-19a8d6baa6e0" containerName="maas-api" Jul 10 18:23:10.548843 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.548822 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.551370 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.551347 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kb8lk\"/\"default-dockercfg-264mv\"" Jul 10 18:23:10.551370 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.551361 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kb8lk\"/\"openshift-service-ca.crt\"" Jul 10 18:23:10.552287 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.552273 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kb8lk\"/\"kube-root-ca.crt\"" Jul 10 18:23:10.558988 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.558952 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf"] Jul 10 18:23:10.617254 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.617225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-sys\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.617678 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.617273 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-podres\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.617678 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.617311 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqqtl\" (UniqueName: \"kubernetes.io/projected/3b977e93-9972-4b37-942d-c53ef73d80bc-kube-api-access-nqqtl\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.617678 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.617343 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-lib-modules\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.617678 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.617397 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-proc\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718167 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718136 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-sys\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718360 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-podres\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718360 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqqtl\" (UniqueName: \"kubernetes.io/projected/3b977e93-9972-4b37-942d-c53ef73d80bc-kube-api-access-nqqtl\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718360 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718269 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-sys\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718529 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718353 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-lib-modules\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718529 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718387 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-proc\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718529 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718354 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-podres\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718529 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718433 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-lib-modules\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.718529 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.718491 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b977e93-9972-4b37-942d-c53ef73d80bc-proc\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.726198 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.726177 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqqtl\" (UniqueName: \"kubernetes.io/projected/3b977e93-9972-4b37-942d-c53ef73d80bc-kube-api-access-nqqtl\") pod \"perf-node-gather-daemonset-gk8jf\" (UID: \"3b977e93-9972-4b37-942d-c53ef73d80bc\") " pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.783573 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.783548 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-g86hh_8f558d18-fa8d-4b98-8610-2427c33f7a4a/download-server/0.log" Jul 10 18:23:10.859824 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.859801 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:10.994890 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:10.994866 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf"] Jul 10 18:23:10.996902 ip-10-0-131-229 kubenswrapper[2568]: W0710 18:23:10.996879 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3b977e93_9972_4b37_942d_c53ef73d80bc.slice/crio-e38ec78b4b4ccd8e815b00f16f66b694748e7ca1247a1662f2f994a3a1cc6225 WatchSource:0}: Error finding container e38ec78b4b4ccd8e815b00f16f66b694748e7ca1247a1662f2f994a3a1cc6225: Status 404 returned error can't find the container with id e38ec78b4b4ccd8e815b00f16f66b694748e7ca1247a1662f2f994a3a1cc6225 Jul 10 18:23:11.890417 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:11.890376 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" event={"ID":"3b977e93-9972-4b37-942d-c53ef73d80bc","Type":"ContainerStarted","Data":"fc82bb2b7b56b8d8ba79fe925fdf4095f1e6cc8ea9aa62c70fc51900e1c01604"} Jul 10 18:23:11.890863 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:11.890440 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" event={"ID":"3b977e93-9972-4b37-942d-c53ef73d80bc","Type":"ContainerStarted","Data":"e38ec78b4b4ccd8e815b00f16f66b694748e7ca1247a1662f2f994a3a1cc6225"} Jul 10 18:23:11.890863 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:11.890668 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:11.905697 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:11.905649 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" podStartSLOduration=1.905635379 podStartE2EDuration="1.905635379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 18:23:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 18:23:11.904579898 +0000 UTC m=+2669.732228667" watchObservedRunningTime="2026-07-10 18:23:11.905635379 +0000 UTC m=+2669.733284196" Jul 10 18:23:12.334254 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:12.334157 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7zqts_4b67c448-4305-4d2b-a46b-b87d9b3860b1/dns/0.log" Jul 10 18:23:12.355885 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:12.355861 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7zqts_4b67c448-4305-4d2b-a46b-b87d9b3860b1/kube-rbac-proxy/0.log" Jul 10 18:23:12.518713 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:12.518689 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lklvj_057ec1b6-2d2e-4a4b-baac-ddd4f33b302a/dns-node-resolver/0.log" Jul 10 18:23:13.116584 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:13.116556 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zg66n_ca928c87-0863-459c-bff1-2c7d3d1dc864/node-ca/0.log" Jul 10 18:23:14.114156 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:14.114131 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tr8vw_ec0a9fc2-7081-44c2-b255-b7f49203c594/discovery/0.log" Jul 10 18:23:14.286690 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:14.286650 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bzllf_9b099279-a74f-4ed1-b641-e5d60c4d037e/payload-processing/0.log" Jul 10 18:23:14.312124 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:14.312099 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77d769cf84-rmlf4_8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c/router/0.log" Jul 10 18:23:14.972379 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:14.972350 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dlx96_c85d73b0-773f-4823-a71b-c8346bc8e11e/serve-healthcheck-canary/0.log" Jul 10 18:23:15.483132 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:15.483103 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-jxbvt_ea8f305d-b4f9-4c84-98b7-ce6ad74f3006/insights-operator/0.log" Jul 10 18:23:15.483523 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:15.483168 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-jxbvt_ea8f305d-b4f9-4c84-98b7-ce6ad74f3006/insights-operator/1.log" Jul 10 18:23:15.575205 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:15.575179 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cvxws_531b0c4e-b6d4-4e2d-bb1a-d36e8572792e/kube-rbac-proxy/0.log" Jul 10 18:23:15.596403 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:15.596383 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cvxws_531b0c4e-b6d4-4e2d-bb1a-d36e8572792e/exporter/0.log" Jul 10 18:23:15.617894 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:15.617869 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cvxws_531b0c4e-b6d4-4e2d-bb1a-d36e8572792e/extractor/0.log" Jul 10 18:23:17.906503 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:17.906477 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kb8lk/perf-node-gather-daemonset-gk8jf" Jul 10 18:23:17.985130 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:17.985099 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56fbcc7474-pd7cs_98424bb4-a006-41b7-9b54-6897d9813843/manager/0.log" Jul 10 18:23:18.192264 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:18.192181 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfdbc665d-tk6xb_cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d/manager/0.log" Jul 10 18:23:18.207871 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:18.207844 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-dfdbc665d-tk6xb_cc8e04e5-d4a1-41fe-bbd8-2bc73684f46d/copy-manifests/0.log" Jul 10 18:23:19.528386 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:19.528312 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7659478ddc-mkqfw_52efb99b-99ac-4b7e-95f9-d4e4f3b85322/manager/0.log" Jul 10 18:23:20.863853 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:20.863781 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tr8vw_ec0a9fc2-7081-44c2-b255-b7f49203c594/discovery/0.log" Jul 10 18:23:20.990056 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:20.990030 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bzllf_9b099279-a74f-4ed1-b641-e5d60c4d037e/payload-processing/0.log" Jul 10 18:23:21.003206 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:21.003182 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77d769cf84-rmlf4_8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c/router/0.log" Jul 10 18:23:22.299277 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:22.299246 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-tr8vw_ec0a9fc2-7081-44c2-b255-b7f49203c594/discovery/0.log" Jul 10 18:23:22.445054 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:22.444952 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bzllf_9b099279-a74f-4ed1-b641-e5d60c4d037e/payload-processing/0.log" Jul 10 18:23:22.458397 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:22.458376 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77d769cf84-rmlf4_8b7f23a3-7a79-4e3a-9e23-3dc9132fa80c/router/0.log" Jul 10 18:23:26.344826 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.344794 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2zbp_0108a2b0-af54-4d1b-804c-21f0fa2e8c61/kube-multus-additional-cni-plugins/0.log" Jul 10 18:23:26.366110 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.366091 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2zbp_0108a2b0-af54-4d1b-804c-21f0fa2e8c61/egress-router-binary-copy/0.log" Jul 10 18:23:26.388932 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.388912 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2zbp_0108a2b0-af54-4d1b-804c-21f0fa2e8c61/cni-plugins/0.log" Jul 10 18:23:26.408610 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.408592 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2zbp_0108a2b0-af54-4d1b-804c-21f0fa2e8c61/bond-cni-plugin/0.log" Jul 10 18:23:26.427867 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.427849 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2zbp_0108a2b0-af54-4d1b-804c-21f0fa2e8c61/routeoverride-cni/0.log" Jul 10 18:23:26.448653 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.448630 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2zbp_0108a2b0-af54-4d1b-804c-21f0fa2e8c61/whereabouts-cni-bincopy/0.log" Jul 10 18:23:26.470403 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.470382 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h2zbp_0108a2b0-af54-4d1b-804c-21f0fa2e8c61/whereabouts-cni/0.log" Jul 10 18:23:26.648566 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.648540 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fpjvn_a763cdaa-9c2e-4c40-adef-857b7ded6ffe/kube-multus/0.log" Jul 10 18:23:26.729333 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.729303 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l5nwq_d209f2ef-b7c8-4817-8950-548fbf79929d/network-metrics-daemon/0.log" Jul 10 18:23:26.747074 ip-10-0-131-229 kubenswrapper[2568]: I0710 18:23:26.747050 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-l5nwq_d209f2ef-b7c8-4817-8950-548fbf79929d/kube-rbac-proxy/0.log"