Jul 09 17:08:58.492499 ip-10-0-135-186 systemd[1]: Starting Kubernetes Kubelet... Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.349948 2586 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358078 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358094 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358098 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358101 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358105 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358107 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358110 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:09:00.413737 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358113 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358115 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358118 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358120 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358123 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358125 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358128 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358131 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358133 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358136 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358138 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358141 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358148 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358151 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358154 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358156 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358159 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358162 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358165 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358168 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:09:00.415853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358170 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358173 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358175 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358180 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358184 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358187 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358190 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358193 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358196 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358199 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358201 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358204 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358207 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358210 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358212 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358215 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358219 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358221 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358224 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:09:00.416690 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358227 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358229 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358231 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358234 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358236 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358240 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358242 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358245 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358247 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358252 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358255 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358259 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358263 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358267 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358270 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358273 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358276 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358279 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:09:00.418550 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358281 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358284 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358287 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358289 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358292 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358296 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358298 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358301 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358304 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358307 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358309 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358312 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358314 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358317 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358319 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358321 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358324 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358326 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358329 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358332 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:09:00.419160 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358335 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358337 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358340 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358342 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358345 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358347 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358350 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358353 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358356 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358359 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358361 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358364 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358366 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358376 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.358379 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359094 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359100 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359103 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359106 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359109 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:09:00.420001 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359111 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359114 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359117 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359120 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359123 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359125 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359128 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359131 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359133 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359139 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359143 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359147 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359150 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359153 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359156 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359159 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359162 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359165 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359167 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:09:00.420596 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359171 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359174 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359176 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359179 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359181 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359184 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359186 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359189 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359191 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359194 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359197 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359199 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359202 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359204 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359207 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359212 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359215 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359218 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359221 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359223 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:09:00.421333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359225 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359228 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359231 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359233 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359236 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359238 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359241 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359243 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359245 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359248 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359250 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359253 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359256 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359259 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359261 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359264 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359266 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359269 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359271 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359274 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:09:00.422215 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359276 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359278 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359281 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359283 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359287 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359291 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359294 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359298 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359302 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359305 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359308 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359310 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359313 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359316 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359319 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359321 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359323 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359326 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359328 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:09:00.424171 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359331 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359334 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359336 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359339 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359341 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359344 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359347 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359349 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359351 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359354 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359356 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359359 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359361 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359364 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359366 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359369 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359783 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359789 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359791 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359794 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:09:00.424811 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359797 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359803 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359806 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359809 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359811 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359814 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359816 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359819 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359821 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359824 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359826 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359829 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359831 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359834 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359836 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359838 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359841 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359843 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359846 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:09:00.427517 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359849 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359851 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359854 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359858 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359861 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359864 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359866 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359869 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359871 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359874 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359876 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359879 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359881 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359884 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359886 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359889 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359892 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359895 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359897 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359900 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:09:00.428125 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359904 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359907 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359911 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359913 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359916 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359919 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359921 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359924 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359926 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359929 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359932 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359934 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359937 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359939 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359943 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359946 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359948 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359951 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359953 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359956 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:09:00.548260 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359958 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:09:00.436504 ip-10-0-135-186 systemd[1]: Started Kubernetes Kubelet. Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359960 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359963 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359965 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359968 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359970 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359973 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359978 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359993 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359996 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.359998 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360001 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360004 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360007 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360009 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360012 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360015 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360017 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360020 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:09:00.561701 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360025 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360028 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360031 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360034 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360036 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360039 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360041 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360046 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360048 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360050 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360054 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360057 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360059 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360062 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360064 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360067 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.360070 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360144 2586 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360152 2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360165 2586 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360175 2586 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 17:09:00.562530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360181 2586 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360192 2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360196 2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360200 2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360203 2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360206 2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360210 2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360213 2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360216 2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360219 2586 flags.go:64] FLAG: --cgroup-root="" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360222 2586 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360225 2586 flags.go:64] FLAG: --client-ca-file="" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360228 2586 flags.go:64] FLAG: --cloud-provider="external" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360231 2586 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360239 2586 flags.go:64] FLAG: --cluster-domain="" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360242 2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360245 2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360248 2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360253 2586 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360256 2586 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360259 2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360262 2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360266 2586 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360269 2586 flags.go:64] FLAG: --contention-profiling="false" Jul 09 17:09:00.563609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360272 2586 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360275 2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360278 2586 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360280 2586 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360284 2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360287 2586 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360290 2586 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360294 2586 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360297 2586 flags.go:64] FLAG: --enable-server="true" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360300 2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360311 2586 flags.go:64] FLAG: --event-burst="100" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360314 2586 flags.go:64] FLAG: --event-qps="50" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360317 2586 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360321 2586 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360324 2586 flags.go:64] FLAG: --eviction-hard="" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360327 2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360330 2586 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360333 2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360336 2586 flags.go:64] FLAG: --eviction-soft="" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360339 2586 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360342 2586 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360345 2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360348 2586 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360350 2586 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360353 2586 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 17:09:00.564328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360356 2586 flags.go:64] FLAG: --feature-gates="" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360361 2586 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360363 2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360367 2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360370 2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360373 2586 flags.go:64] FLAG: --healthz-port="10248" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360376 2586 flags.go:64] FLAG: --help="false" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360378 2586 flags.go:64] FLAG: --hostname-override="ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360381 2586 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360384 2586 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360387 2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360390 2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360394 2586 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360396 2586 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360400 2586 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360403 2586 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360406 2586 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360409 2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360418 2586 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360421 2586 flags.go:64] FLAG: --kube-reserved="" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360424 2586 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360427 2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360430 2586 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360432 2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 17:09:00.565182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360447 2586 flags.go:64] FLAG: --lock-file="" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360451 2586 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360454 2586 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360457 2586 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360462 2586 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360465 2586 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360468 2586 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360470 2586 flags.go:64] FLAG: --logging-format="text" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360473 2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360477 2586 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360480 2586 flags.go:64] FLAG: --manifest-url="" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360483 2586 flags.go:64] FLAG: --manifest-url-header="" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360491 2586 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360494 2586 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360498 2586 flags.go:64] FLAG: --max-pods="110" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360501 2586 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360504 2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360507 2586 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360510 2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360513 2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360516 2586 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360519 2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360528 2586 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360531 2586 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360534 2586 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 17:09:00.566051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360537 2586 flags.go:64] FLAG: --pod-cidr="" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360540 2586 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360552 2586 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360555 2586 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360558 2586 flags.go:64] FLAG: --pods-per-core="0" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360561 2586 flags.go:64] FLAG: --port="10250" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360563 2586 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360567 2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0616bc1a85d577cb2" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360570 2586 flags.go:64] FLAG: --qos-reserved="" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360573 2586 flags.go:64] FLAG: --read-only-port="10255" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360576 2586 flags.go:64] FLAG: --register-node="true" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360578 2586 flags.go:64] FLAG: --register-with-taints="" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360582 2586 flags.go:64] FLAG: --registry-burst="10" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360585 2586 flags.go:64] FLAG: --registry-qps="5" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360588 2586 flags.go:64] FLAG: --reserved-cpus="" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360591 2586 flags.go:64] FLAG: --reserved-memory="" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360595 2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360598 2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360601 2586 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360604 2586 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360607 2586 flags.go:64] FLAG: --runonce="false" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360610 2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360613 2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360616 2586 flags.go:64] FLAG: --seccomp-default="false" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360619 2586 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 17:09:00.566908 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360622 2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360624 2586 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360628 2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360631 2586 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360634 2586 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360638 2586 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360641 2586 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360644 2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360647 2586 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360650 2586 flags.go:64] FLAG: --system-cgroups="" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360656 2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360661 2586 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360664 2586 flags.go:64] FLAG: --tls-cert-file="" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.360667 2586 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361167 2586 flags.go:64] FLAG: --tls-min-version="" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361173 2586 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361177 2586 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361182 2586 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361186 2586 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361189 2586 flags.go:64] FLAG: --v="2" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361193 2586 flags.go:64] FLAG: --version="false" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361197 2586 flags.go:64] FLAG: --vmodule="" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361201 2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.361282 2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 17:09:00.569206 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361505 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361510 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361514 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361516 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361519 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361522 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361524 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361527 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361529 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361532 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361535 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361537 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361540 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361545 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361547 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361550 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361553 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361555 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361558 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361561 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:09:00.570048 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361564 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361566 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361569 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361572 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361574 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361577 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361579 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361582 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361584 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361587 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361589 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361593 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361597 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361600 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361604 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361607 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361610 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361613 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361615 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:09:00.682012 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361618 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361620 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361623 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361625 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361627 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361630 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361633 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361636 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361639 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361641 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361644 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361646 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361649 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361653 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361655 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361658 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361661 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361663 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361666 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361669 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:09:00.682652 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361671 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361673 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361676 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361678 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361682 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361685 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361688 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361690 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361693 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361695 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361698 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361700 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361703 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361705 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361708 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361710 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361713 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361715 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361719 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:09:00.683932 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361721 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361724 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361726 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361730 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361732 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361735 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361738 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361746 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361749 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361752 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361754 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361758 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361762 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361765 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361767 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361770 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361772 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361776 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361779 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361781 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:09:00.684477 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.361784 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.362408 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.371378 2586 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.371396 2586 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.372207 2586 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.372216 2586 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372280 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372286 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372289 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372293 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372297 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372300 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372303 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372306 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372309 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:09:00.685019 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372312 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372315 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372317 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372320 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372323 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372325 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372328 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372331 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372333 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372336 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372339 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372341 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372344 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372347 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372350 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372354 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372358 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372360 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372363 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372366 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:09:00.810157 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372369 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372372 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372374 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372377 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372379 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372382 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372384 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372387 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372389 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372392 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372394 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372397 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372399 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372402 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372404 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372407 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372410 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372414 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372418 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:09:00.811021 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372420 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372423 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372426 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372429 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372432 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372447 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372450 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372453 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372456 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372459 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372462 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372465 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372468 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372471 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372473 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372476 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372479 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372482 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372484 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:09:00.811842 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372487 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372489 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372492 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372494 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372497 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372499 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372502 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372505 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372507 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372510 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372512 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372515 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372525 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372529 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372532 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372535 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372538 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372543 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372546 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372548 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:09:00.812705 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372553 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372556 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372558 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372561 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372564 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372566 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372569 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372571 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372574 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372577 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372579 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372582 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.372587 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372701 2586 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372706 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:09:00.813776 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372709 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372712 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372715 2586 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372718 2586 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372721 2586 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372724 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372726 2586 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372729 2586 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372731 2586 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372734 2586 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372742 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372745 2586 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372748 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372750 2586 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372753 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372757 2586 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372760 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372764 2586 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372767 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372769 2586 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:09:00.814892 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372772 2586 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372774 2586 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372777 2586 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372780 2586 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372782 2586 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372785 2586 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372788 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372790 2586 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372793 2586 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372796 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372798 2586 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372801 2586 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372803 2586 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372806 2586 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372809 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372811 2586 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372814 2586 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372816 2586 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372819 2586 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:09:00.815424 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372821 2586 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372824 2586 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372826 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372828 2586 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372837 2586 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372840 2586 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372842 2586 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372845 2586 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372849 2586 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372852 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372856 2586 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372858 2586 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372861 2586 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372863 2586 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372866 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372869 2586 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372871 2586 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372874 2586 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372876 2586 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:09:00.815964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372879 2586 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372881 2586 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372884 2586 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372886 2586 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372889 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372892 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372894 2586 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372897 2586 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372900 2586 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372902 2586 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372905 2586 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372908 2586 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372912 2586 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372915 2586 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372918 2586 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372921 2586 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372924 2586 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372927 2586 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372930 2586 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372933 2586 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:09:00.816427 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372935 2586 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372939 2586 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372941 2586 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372945 2586 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372948 2586 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372950 2586 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372953 2586 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372955 2586 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372959 2586 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372962 2586 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372965 2586 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372968 2586 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372971 2586 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372973 2586 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372976 2586 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372979 2586 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372982 2586 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372984 2586 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:09:00.817241 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:00.372987 2586 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.372992 2586 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.373097 2586 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.377192 2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.378634 2586 server.go:1019] "Starting client certificate rotation" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.380893 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.380928 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.381795 2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.385138 2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.394842 2586 log.go:25] "Validated CRI v1 runtime API" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.397566 2586 log.go:25] "Validated CRI v1 image API" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.398364 2586 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 17:09:00.818008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.403987 2586 fs.go:135] Filesystem UUIDs: map[026134fe-f5ac-4ad3-acb7-0f5bcf7bad03:/dev/nvme0n1p3 1f9a2873-8467-4d90-afa5-ab086211452d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 09 17:09:00.818568 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.404013 2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 17:09:00.818621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.411626 2586 manager.go:216] Machine: {Timestamp:2026-07-09 17:09:00.410913396 +0000 UTC m=+0.340358901 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3045822 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22024ade188165a64038e70188ac7d SystemUUID:ec22024a-de18-8165-a640-38e70188ac7d BootID:504d2c46-57df-4ee9-8dd9-3e418b7e4b05 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0f:9d:de:78:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0f:9d:de:78:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:59:d9:48:d7:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 17:09:00.818621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.411762 2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 17:09:00.818621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.411856 2586 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 17:09:00.818621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.413140 2586 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 17:09:00.818621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.413164 2586 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-186.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 17:09:00.818621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.414535 2586 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 17:09:00.818621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.414548 2586 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 17:09:00.818621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.414558 2586 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.415201 2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.416040 2586 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.420583 2586 state_mem.go:36] "Initialized new in-memory state store" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.420697 2586 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.421372 2586 kubelet.go:486] "Attempting to sync node with API server" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.421382 2586 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.421397 2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.421405 2586 kubelet.go:398] "Adding apiserver pod source" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.421416 2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.421847 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.421865 2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.423986 2586 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.429384 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.430158 2586 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.430206 2586 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.430961 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.430984 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.430994 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431002 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431014 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431023 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431052 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431058 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431065 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431070 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431084 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431092 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431851 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.431867 2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.435576 2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-186.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.435628 2586 server.go:1268] "Started kubelet" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.435644 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.435705 2586 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.435741 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.435789 2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.435852 2586 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.437851 2586 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.438418 2586 server.go:310] "Adding debug handlers to kubelet server" Jul 09 17:09:00.819033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.441346 2586 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.441693 2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.444170 2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-186.ec2.internal.18c0ae284608115c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-186.ec2.internal,UID:ip-10-0-135-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-186.ec2.internal,},FirstTimestamp:2026-07-09 17:09:00.435575132 +0000 UTC m=+0.365020635,LastTimestamp:2026-07-09 17:09:00.435575132 +0000 UTC m=+0.365020635,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-186.ec2.internal,}" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.446473 2586 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.446494 2586 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.446595 2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.446643 2586 reconstruct.go:97] "Volume reconstruction finished" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.446650 2586 reconciler.go:29] "Reconciler: start to sync state" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.447740 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.448044 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zcckh" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.449398 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zcckh" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.449476 2586 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.450463 2586 factory.go:55] Registering systemd factory Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.450478 2586 factory.go:223] Registration of the systemd container factory successfully Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.450726 2586 factory.go:153] Registering CRI-O factory Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.450741 2586 factory.go:223] Registration of the crio container factory successfully Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.450781 2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.450806 2586 factory.go:103] Registering Raw factory Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.450816 2586 manager.go:1196] Started watching for new ooms in manager Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.451373 2586 manager.go:349] Starting recovery of all containers Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.456717 2586 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.458995 2586 manager.go:354] Recovery completed Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.459635 2586 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.465614 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.467701 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.467724 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.467739 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.468261 2586 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.468270 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.468271 2586 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.468367 2586 state_mem.go:36] "Initialized new in-memory state store" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.468988 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:09:00.820381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.469012 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.469023 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.469617 2586 policy_none.go:49] "None policy: Start" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.469631 2586 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.469644 2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.470615 2586 policy_none.go:47] "Start" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.502118 2586 manager.go:337] "Starting Device Plugin manager" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.502291 2586 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.502303 2586 server.go:87] "Starting device plugin registration server" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.504791 2586 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.505006 2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.505525 2586 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.505626 2586 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.505635 2586 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.509287 2586 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.509355 2586 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.605842 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.606993 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.607042 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.607053 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.607105 2586 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.615579 2586 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.615607 2586 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-186.ec2.internal\": node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.633581 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.663726 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.664998 2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.665037 2586 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.665059 2586 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.665093 2586 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.670024 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.734071 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.765538 2586 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal"] Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.765624 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.767238 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.767264 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.767274 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.768550 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.768666 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.821226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.768694 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.769278 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.769281 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.769299 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.769309 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.769319 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.769332 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.771222 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.771249 2586 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.771897 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.771914 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:09:00.822203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.771923 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:09:00.823702 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.823677 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-186.ec2.internal\" not found" node="ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.827734 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.827713 2586 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-186.ec2.internal\" not found" node="ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.839033 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.839008 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:00.865514 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.865479 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b45ab77f7adac80ae4f46c80d27c83b4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-186.ec2.internal\" (UID: \"b45ab77f7adac80ae4f46c80d27c83b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.865514 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.865517 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.865707 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.865542 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.940057 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:00.939979 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:00.966505 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.966469 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.966600 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.966510 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.966600 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.966534 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b45ab77f7adac80ae4f46c80d27c83b4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-186.ec2.internal\" (UID: \"b45ab77f7adac80ae4f46c80d27c83b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.966600 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.966566 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.966600 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.966577 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aa7a67259a3462e5ba6775b001f3248-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal\" (UID: \"3aa7a67259a3462e5ba6775b001f3248\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:00.966735 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:00.966616 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b45ab77f7adac80ae4f46c80d27c83b4-config\") pod \"kube-apiserver-proxy-ip-10-0-135-186.ec2.internal\" (UID: \"b45ab77f7adac80ae4f46c80d27c83b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" Jul 09 17:09:01.040609 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:01.040576 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:01.127156 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.127118 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:01.131635 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.131216 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" Jul 09 17:09:01.140849 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:01.140815 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:01.241527 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:01.241422 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:01.342026 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:01.341990 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:01.381587 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.381538 2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 17:09:01.381762 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.381705 2586 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 17:09:01.442917 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:01.442885 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:01.453647 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.453612 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 17:04:00 +0000 UTC" deadline="2028-04-23 09:58:24.425246626 +0000 UTC" Jul 09 17:09:01.453647 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.453641 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15688h49m22.971608856s" Jul 09 17:09:01.543982 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:01.543914 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:01.644536 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:01.644488 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:01.676797 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:01.676764 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aa7a67259a3462e5ba6775b001f3248.slice/crio-2a3773db8b0bbdac95d5b742ae5d6cf83d2f01f5d44cc73dcf491fab60b1eee5 WatchSource:0}: Error finding container 2a3773db8b0bbdac95d5b742ae5d6cf83d2f01f5d44cc73dcf491fab60b1eee5: Status 404 returned error can't find the container with id 2a3773db8b0bbdac95d5b742ae5d6cf83d2f01f5d44cc73dcf491fab60b1eee5 Jul 09 17:09:01.677226 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:01.677211 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45ab77f7adac80ae4f46c80d27c83b4.slice/crio-26defea502d3928f86fb619f611fcd7aac37a11cfc4357f28730d75fb77023cb WatchSource:0}: Error finding container 26defea502d3928f86fb619f611fcd7aac37a11cfc4357f28730d75fb77023cb: Status 404 returned error can't find the container with id 26defea502d3928f86fb619f611fcd7aac37a11cfc4357f28730d75fb77023cb Jul 09 17:09:01.681721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.681708 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:09:01.745638 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:01.745593 2586 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-186.ec2.internal\" not found" Jul 09 17:09:01.770959 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.770933 2586 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:09:01.848636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.848558 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" Jul 09 17:09:01.859016 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.858994 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 17:09:01.860641 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.860625 2586 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" Jul 09 17:09:01.871168 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.871145 2586 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 17:09:01.968468 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.968429 2586 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:09:01.981376 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:01.981352 2586 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:09:02.422336 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.422305 2586 apiserver.go:52] "Watching apiserver" Jul 09 17:09:02.433274 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.433241 2586 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 17:09:02.433846 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.433814 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-qwprr","openshift-network-diagnostics/network-check-target-4nvpb","kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal","openshift-multus/multus-tklf4","openshift-multus/network-metrics-daemon-hbwvp","openshift-network-operator/iptables-alerter-d46qp","openshift-ovn-kubernetes/ovnkube-node-rcs4h","kube-system/global-pull-secret-syncer-txnlq","kube-system/konnectivity-agent-l448f","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x","openshift-cluster-node-tuning-operator/tuned-r6bmz","openshift-dns/node-resolver-kshcf","openshift-image-registry/node-ca-5x9qp"] Jul 09 17:09:02.435784 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.435761 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:02.437781 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.437018 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.438433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.438412 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.439293 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.439272 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 17:09:02.439563 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.439545 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xcgzj\"" Jul 09 17:09:02.439772 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.439753 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 17:09:02.440176 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.440152 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cltd2\"" Jul 09 17:09:02.440650 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.440634 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.441912 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.441891 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 17:09:02.444238 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.444217 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 17:09:02.447267 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.445168 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p8zcn\"" Jul 09 17:09:02.447267 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.445373 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 17:09:02.447267 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.445594 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:02.447267 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.445885 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 17:09:02.447267 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.446641 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:02.447267 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.446711 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gmfjq\"" Jul 09 17:09:02.447267 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.446767 2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 17:09:02.451465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.449110 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.451465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.449496 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.451465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.450020 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.451465 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.450088 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:02.451465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.450457 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 17:09:02.453849 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.452892 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 17:09:02.453849 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.453595 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 17:09:02.453849 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.453730 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 17:09:02.454940 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.454849 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:02.456769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.456000 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 17:09:02.456769 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.456469 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:02.456769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.456687 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 17:09:02.457312 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.457292 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.457958 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.457734 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 17:09:02.457958 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.457741 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mzt5m\"" Jul 09 17:09:02.458342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.458200 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 17:09:02.458342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.458259 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 17:09:02.458633 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.458614 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 17:09:02.458728 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.458710 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:09:02.458850 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.458834 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 17:09:02.458950 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.458920 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kqbnz\"" Jul 09 17:09:02.459391 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.459140 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 17:09:02.459391 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.459270 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 17:09:02.460079 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.460064 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hzftv\"" Jul 09 17:09:02.460079 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.460065 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 17:09:02.460255 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.460237 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 17:09:02.461675 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.461118 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.462709 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.462687 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.464064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.463766 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 17:09:02.464064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.463880 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sltzb\"" Jul 09 17:09:02.464064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.464025 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 17:09:02.464256 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.464239 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 17:09:02.465412 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.465394 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:09:02.465943 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.465797 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 17:09:02.466030 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.465973 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tmdp5\"" Jul 09 17:09:02.470489 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.470468 2586 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474158 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-systemd-units\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474193 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-systemd\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474219 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-cni-netd\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474243 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-multus-certs\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474268 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-run-netns\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474302 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474330 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbf9d27e-b21d-4436-8502-3543544a2fe1-hosts-file\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474358 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-kubelet-config\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474410 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndjck\" (UniqueName: \"kubernetes.io/projected/35d602c3-6358-40e3-a13a-35f2a17c273d-kube-api-access-ndjck\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474434 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-slash\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474476 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474507 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-env-overrides\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474529 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkdn8\" (UniqueName: \"kubernetes.io/projected/ea9dd5df-3bc5-4eb7-a428-a93987879809-kube-api-access-wkdn8\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:02.475327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474551 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b238ba72-80c9-42b3-a046-cc4cc546e62d-host\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474583 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-var-lib-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474613 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-cni-multus\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474635 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-kubelet\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474667 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-cni-bin\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474690 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovnkube-config\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474713 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9204fadf-bf81-4ebb-bf8f-75b94fa2db53-konnectivity-ca\") pod \"konnectivity-agent-l448f\" (UID: \"9204fadf-bf81-4ebb-bf8f-75b94fa2db53\") " pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474736 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-node-log\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474756 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-log-socket\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474785 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqdqn\" (UniqueName: \"kubernetes.io/projected/0c29139d-04e4-45ee-a125-a01247b1f99e-kube-api-access-sqdqn\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474809 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-ovn\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474835 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovn-node-metrics-cert\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474858 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474901 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:02.476064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474924 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-kubelet\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474952 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474975 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-etc-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.474998 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-system-cni-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475023 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-socket-dir-parent\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475047 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkk5d\" (UniqueName: \"kubernetes.io/projected/dbf9d27e-b21d-4436-8502-3543544a2fe1-kube-api-access-wkk5d\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475080 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35d602c3-6358-40e3-a13a-35f2a17c273d-host-slash\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475107 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9204fadf-bf81-4ebb-bf8f-75b94fa2db53-agent-certs\") pod \"konnectivity-agent-l448f\" (UID: \"9204fadf-bf81-4ebb-bf8f-75b94fa2db53\") " pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475296 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c29139d-04e4-45ee-a125-a01247b1f99e-cni-binary-copy\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475382 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-k8s-cni-cncf-io\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475431 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf9d27e-b21d-4436-8502-3543544a2fe1-tmp-dir\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475495 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovnkube-script-lib\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475526 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-dbus\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475556 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:02.476898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475586 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b238ba72-80c9-42b3-a046-cc4cc546e62d-serviceca\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475610 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-os-release\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475632 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-conf-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475665 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxsvc\" (UniqueName: \"kubernetes.io/projected/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-kube-api-access-pxsvc\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475708 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-cni-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475733 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-cnibin\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475762 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-cni-bin\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475786 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-hostroot\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475818 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35d602c3-6358-40e3-a13a-35f2a17c273d-iptables-alerter-script\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475868 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zmjg\" (UniqueName: \"kubernetes.io/projected/b238ba72-80c9-42b3-a046-cc4cc546e62d-kube-api-access-4zmjg\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475904 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-netns\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475952 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-daemon-config\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.477555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.475976 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-etc-kubernetes\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.488298 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.488277 2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w5rlf" Jul 09 17:09:02.499734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.499385 2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w5rlf" Jul 09 17:09:02.547596 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.547520 2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 17:09:02.576262 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576154 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-etc-kubernetes\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.576262 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576197 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58064e17-6c23-4e5f-991b-37c9969558c6-tmp\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.576262 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576222 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-systemd-units\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576289 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-etc-kubernetes\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576296 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-systemd-units\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576328 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-systemd\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576356 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-cni-netd\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576414 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-multus-certs\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576383 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-systemd\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576454 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576483 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj7gj\" (UniqueName: \"kubernetes.io/projected/13e9a0cf-c2b6-4824-8447-ec09c276fd53-kube-api-access-sj7gj\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576508 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-run-netns\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576533 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576540 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-cni-netd\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576556 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbf9d27e-b21d-4436-8502-3543544a2fe1-hosts-file\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576506 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-multus-certs\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.576678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576581 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-kubelet-config\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576590 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-run-netns\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576605 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndjck\" (UniqueName: \"kubernetes.io/projected/35d602c3-6358-40e3-a13a-35f2a17c273d-kube-api-access-ndjck\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576611 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576665 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbf9d27e-b21d-4436-8502-3543544a2fe1-hosts-file\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576676 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-kubelet-config\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576695 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-slash\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576726 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-slash\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576725 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576762 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576780 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-env-overrides\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576821 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkdn8\" (UniqueName: \"kubernetes.io/projected/ea9dd5df-3bc5-4eb7-a428-a93987879809-kube-api-access-wkdn8\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576844 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b238ba72-80c9-42b3-a046-cc4cc546e62d-host\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576898 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cni-binary-copy\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.577387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576927 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-var-lib-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576951 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-cni-multus\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576975 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-system-cni-dir\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.576998 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-modprobe-d\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577024 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-kubelet\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577048 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-cni-bin\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577083 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovnkube-config\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577107 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-var-lib-kubelet\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577108 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-cni-multus\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577129 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-host\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577154 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9204fadf-bf81-4ebb-bf8f-75b94fa2db53-konnectivity-ca\") pod \"konnectivity-agent-l448f\" (UID: \"9204fadf-bf81-4ebb-bf8f-75b94fa2db53\") " pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577157 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b238ba72-80c9-42b3-a046-cc4cc546e62d-host\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577176 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-node-log\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577200 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-log-socket\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577207 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-var-lib-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577226 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqdqn\" (UniqueName: \"kubernetes.io/projected/0c29139d-04e4-45ee-a125-a01247b1f99e-kube-api-access-sqdqn\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577254 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577278 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-sys\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577302 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-ovn\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577315 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-env-overrides\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577324 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovn-node-metrics-cert\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577350 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-device-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577390 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577397 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-log-socket\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577418 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577504 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-kubelet\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577509 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-kubelet\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577507 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.578580 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577563 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-node-log\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577654 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-run-ovn\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577666 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cnibin\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577693 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-sys-fs\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577697 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-kubelet\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577741 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gl6b\" (UniqueName: \"kubernetes.io/projected/866a6276-35db-41bf-abc1-cdcf85a671cc-kube-api-access-7gl6b\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577753 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-host-cni-bin\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577764 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovnkube-config\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577777 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577807 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-etc-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577858 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-system-cni-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577896 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-etc-openvswitch\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577893 2586 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.577916 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577943 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-socket-dir-parent\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.579124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577950 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-system-cni-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.577985 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-os-release\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578005 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-socket-dir-parent\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578013 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkk5d\" (UniqueName: \"kubernetes.io/projected/dbf9d27e-b21d-4436-8502-3543544a2fe1-kube-api-access-wkk5d\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.578082 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret podName:c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:03.078060526 +0000 UTC m=+3.007506032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret") pod "global-pull-secret-syncer-txnlq" (UID: "c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578100 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35d602c3-6358-40e3-a13a-35f2a17c273d-host-slash\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578126 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9204fadf-bf81-4ebb-bf8f-75b94fa2db53-konnectivity-ca\") pod \"konnectivity-agent-l448f\" (UID: \"9204fadf-bf81-4ebb-bf8f-75b94fa2db53\") " pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578139 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9204fadf-bf81-4ebb-bf8f-75b94fa2db53-agent-certs\") pod \"konnectivity-agent-l448f\" (UID: \"9204fadf-bf81-4ebb-bf8f-75b94fa2db53\") " pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578166 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c29139d-04e4-45ee-a125-a01247b1f99e-cni-binary-copy\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578176 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35d602c3-6358-40e3-a13a-35f2a17c273d-host-slash\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578208 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-k8s-cni-cncf-io\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578294 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-k8s-cni-cncf-io\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578287 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578340 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf9d27e-b21d-4436-8502-3543544a2fe1-tmp-dir\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.579688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578386 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovnkube-script-lib\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578415 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578457 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-registration-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578480 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysctl-conf\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578586 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-run\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578616 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-dbus\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578656 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578678 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-dbus\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578691 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b238ba72-80c9-42b3-a046-cc4cc546e62d-serviceca\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578716 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-os-release\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578740 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-conf-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.578759 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578767 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578794 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysctl-d\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.580396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578798 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c29139d-04e4-45ee-a125-a01247b1f99e-cni-binary-copy\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.578814 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs podName:ea9dd5df-3bc5-4eb7-a428-a93987879809 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:03.078797297 +0000 UTC m=+3.008242792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs") pod "network-metrics-daemon-hbwvp" (UID: "ea9dd5df-3bc5-4eb7-a428-a93987879809") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578856 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-systemd\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578885 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxsvc\" (UniqueName: \"kubernetes.io/projected/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-kube-api-access-pxsvc\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578890 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-conf-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578919 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-cni-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578946 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-cnibin\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578969 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-cni-bin\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578984 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovnkube-script-lib\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578991 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-hostroot\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.578860 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-os-release\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579028 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-lib-modules\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579051 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58064e17-6c23-4e5f-991b-37c9969558c6-etc-tuned\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579070 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-cnibin\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579087 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35d602c3-6358-40e3-a13a-35f2a17c273d-iptables-alerter-script\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579095 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-hostroot\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579108 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-socket-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579129 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-cni-dir\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579126 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-var-lib-cni-bin\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579136 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysconfig\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579178 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-kubernetes\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579206 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbvsh\" (UniqueName: \"kubernetes.io/projected/58064e17-6c23-4e5f-991b-37c9969558c6-kube-api-access-nbvsh\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579238 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zmjg\" (UniqueName: \"kubernetes.io/projected/b238ba72-80c9-42b3-a046-cc4cc546e62d-kube-api-access-4zmjg\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579240 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf9d27e-b21d-4436-8502-3543544a2fe1-tmp-dir\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579263 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-netns\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579295 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0c29139d-04e4-45ee-a125-a01247b1f99e-host-run-netns\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579303 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-daemon-config\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579618 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35d602c3-6358-40e3-a13a-35f2a17c273d-iptables-alerter-script\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579759 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0c29139d-04e4-45ee-a125-a01247b1f99e-multus-daemon-config\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.581733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.579909 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b238ba72-80c9-42b3-a046-cc4cc546e62d-serviceca\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.582807 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.582780 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-ovn-node-metrics-cert\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.582939 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.582897 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9204fadf-bf81-4ebb-bf8f-75b94fa2db53-agent-certs\") pod \"konnectivity-agent-l448f\" (UID: \"9204fadf-bf81-4ebb-bf8f-75b94fa2db53\") " pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:02.588842 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.588782 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkdn8\" (UniqueName: \"kubernetes.io/projected/ea9dd5df-3bc5-4eb7-a428-a93987879809-kube-api-access-wkdn8\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:02.588842 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.588829 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:09:02.589022 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.588853 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:09:02.589022 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.588866 2586 projected.go:196] Error preparing data for projected volume kube-api-access-tmddh for pod openshift-network-diagnostics/network-check-target-4nvpb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:02.589022 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:02.588934 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh podName:66284223-a2b7-4e40-978f-f44828b75e41 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:03.08891535 +0000 UTC m=+3.018360843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tmddh" (UniqueName: "kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh") pod "network-check-target-4nvpb" (UID: "66284223-a2b7-4e40-978f-f44828b75e41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:02.593024 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.592998 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkk5d\" (UniqueName: \"kubernetes.io/projected/dbf9d27e-b21d-4436-8502-3543544a2fe1-kube-api-access-wkk5d\") pod \"node-resolver-kshcf\" (UID: \"dbf9d27e-b21d-4436-8502-3543544a2fe1\") " pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.593587 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.593530 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndjck\" (UniqueName: \"kubernetes.io/projected/35d602c3-6358-40e3-a13a-35f2a17c273d-kube-api-access-ndjck\") pod \"iptables-alerter-d46qp\" (UID: \"35d602c3-6358-40e3-a13a-35f2a17c273d\") " pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.593868 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.593803 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqdqn\" (UniqueName: \"kubernetes.io/projected/0c29139d-04e4-45ee-a125-a01247b1f99e-kube-api-access-sqdqn\") pod \"multus-tklf4\" (UID: \"0c29139d-04e4-45ee-a125-a01247b1f99e\") " pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.595361 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.595295 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zmjg\" (UniqueName: \"kubernetes.io/projected/b238ba72-80c9-42b3-a046-cc4cc546e62d-kube-api-access-4zmjg\") pod \"node-ca-5x9qp\" (UID: \"b238ba72-80c9-42b3-a046-cc4cc546e62d\") " pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.595921 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.595894 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxsvc\" (UniqueName: \"kubernetes.io/projected/7cdfdc66-e339-48f1-bdb0-d930d999d0fc-kube-api-access-pxsvc\") pod \"ovnkube-node-rcs4h\" (UID: \"7cdfdc66-e339-48f1-bdb0-d930d999d0fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.670266 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.670216 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" event={"ID":"3aa7a67259a3462e5ba6775b001f3248","Type":"ContainerStarted","Data":"2a3773db8b0bbdac95d5b742ae5d6cf83d2f01f5d44cc73dcf491fab60b1eee5"} Jul 09 17:09:02.671408 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.671380 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" event={"ID":"b45ab77f7adac80ae4f46c80d27c83b4","Type":"ContainerStarted","Data":"26defea502d3928f86fb619f611fcd7aac37a11cfc4357f28730d75fb77023cb"} Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680027 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680079 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysctl-d\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680107 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-systemd\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680625 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-lib-modules\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680667 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58064e17-6c23-4e5f-991b-37c9969558c6-etc-tuned\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680702 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-socket-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680733 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysconfig\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680761 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-kubernetes\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680777 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysctl-d\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680793 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbvsh\" (UniqueName: \"kubernetes.io/projected/58064e17-6c23-4e5f-991b-37c9969558c6-kube-api-access-nbvsh\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.680934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.680846 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58064e17-6c23-4e5f-991b-37c9969558c6-tmp\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681609 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-systemd\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681707 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysconfig\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681752 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-socket-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681776 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-kubernetes\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681865 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681884 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-lib-modules\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681882 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681931 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj7gj\" (UniqueName: \"kubernetes.io/projected/13e9a0cf-c2b6-4824-8447-ec09c276fd53-kube-api-access-sj7gj\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.681979 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cni-binary-copy\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682014 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-system-cni-dir\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682045 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682048 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-modprobe-d\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682098 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-var-lib-kubelet\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.682382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682126 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-host\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682151 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-modprobe-d\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682195 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682214 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-system-cni-dir\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682225 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-sys\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682277 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-host\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682295 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-var-lib-kubelet\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682353 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-sys\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682551 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cni-binary-copy\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682652 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-device-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682716 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cnibin\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682720 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-device-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682752 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-sys-fs\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682785 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gl6b\" (UniqueName: \"kubernetes.io/projected/866a6276-35db-41bf-abc1-cdcf85a671cc-kube-api-access-7gl6b\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682867 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cnibin\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682873 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-os-release\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682911 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-sys-fs\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682922 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682963 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.682997 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/13e9a0cf-c2b6-4824-8447-ec09c276fd53-os-release\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683019 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-registration-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683054 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysctl-conf\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683062 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683077 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-run\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683198 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683229 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/866a6276-35db-41bf-abc1-cdcf85a671cc-registration-dir\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683229 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-run\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683341 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58064e17-6c23-4e5f-991b-37c9969558c6-etc-sysctl-conf\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.683790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.683464 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/13e9a0cf-c2b6-4824-8447-ec09c276fd53-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.684504 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.684464 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58064e17-6c23-4e5f-991b-37c9969558c6-etc-tuned\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.685229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.685208 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58064e17-6c23-4e5f-991b-37c9969558c6-tmp\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.690359 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.690335 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbvsh\" (UniqueName: \"kubernetes.io/projected/58064e17-6c23-4e5f-991b-37c9969558c6-kube-api-access-nbvsh\") pod \"tuned-r6bmz\" (UID: \"58064e17-6c23-4e5f-991b-37c9969558c6\") " pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.690720 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.690697 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj7gj\" (UniqueName: \"kubernetes.io/projected/13e9a0cf-c2b6-4824-8447-ec09c276fd53-kube-api-access-sj7gj\") pod \"multus-additional-cni-plugins-qwprr\" (UID: \"13e9a0cf-c2b6-4824-8447-ec09c276fd53\") " pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.691612 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.691593 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gl6b\" (UniqueName: \"kubernetes.io/projected/866a6276-35db-41bf-abc1-cdcf85a671cc-kube-api-access-7gl6b\") pod \"aws-ebs-csi-driver-node-dlr6x\" (UID: \"866a6276-35db-41bf-abc1-cdcf85a671cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.762942 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.762901 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:02.770561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.770526 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kshcf" Jul 09 17:09:02.772765 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:02.772727 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9204fadf_bf81_4ebb_bf8f_75b94fa2db53.slice/crio-fb01ad7b2f40f4291fad9497ad65bb6d9309b72f97ce1da473f4200f752f1475 WatchSource:0}: Error finding container fb01ad7b2f40f4291fad9497ad65bb6d9309b72f97ce1da473f4200f752f1475: Status 404 returned error can't find the container with id fb01ad7b2f40f4291fad9497ad65bb6d9309b72f97ce1da473f4200f752f1475 Jul 09 17:09:02.779333 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:02.779299 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf9d27e_b21d_4436_8502_3543544a2fe1.slice/crio-d178be831fca45d193f052ef0b94733ef24502c8f419887109e90f17448264b7 WatchSource:0}: Error finding container d178be831fca45d193f052ef0b94733ef24502c8f419887109e90f17448264b7: Status 404 returned error can't find the container with id d178be831fca45d193f052ef0b94733ef24502c8f419887109e90f17448264b7 Jul 09 17:09:02.780689 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.780666 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5x9qp" Jul 09 17:09:02.788283 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.788262 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:02.797141 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.797117 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tklf4" Jul 09 17:09:02.805244 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.805221 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d46qp" Jul 09 17:09:02.813512 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.813479 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qwprr" Jul 09 17:09:02.821366 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.821211 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" Jul 09 17:09:02.831039 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.831019 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" Jul 09 17:09:02.868257 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:02.868226 2586 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:09:03.085672 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.085576 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:03.085672 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.085646 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:03.085903 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.085694 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:03.085903 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.085738 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:03.085903 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.085763 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret podName:c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:04.085745134 +0000 UTC m=+4.015190650 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret") pod "global-pull-secret-syncer-txnlq" (UID: "c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:03.085903 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.085783 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs podName:ea9dd5df-3bc5-4eb7-a428-a93987879809 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:04.085771997 +0000 UTC m=+4.015217486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs") pod "network-metrics-daemon-hbwvp" (UID: "ea9dd5df-3bc5-4eb7-a428-a93987879809") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:03.114741 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:03.114707 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb238ba72_80c9_42b3_a046_cc4cc546e62d.slice/crio-f83c4eba5ba73b1274b86cf26c971817e36a9b5eb20c4f40c80d1170b42ffa05 WatchSource:0}: Error finding container f83c4eba5ba73b1274b86cf26c971817e36a9b5eb20c4f40c80d1170b42ffa05: Status 404 returned error can't find the container with id f83c4eba5ba73b1274b86cf26c971817e36a9b5eb20c4f40c80d1170b42ffa05 Jul 09 17:09:03.186603 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.186569 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:03.186776 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.186758 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:09:03.186848 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.186780 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:09:03.186848 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.186791 2586 projected.go:196] Error preparing data for projected volume kube-api-access-tmddh for pod openshift-network-diagnostics/network-check-target-4nvpb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:03.186941 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.186852 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh podName:66284223-a2b7-4e40-978f-f44828b75e41 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:04.186833276 +0000 UTC m=+4.116278789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tmddh" (UniqueName: "kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh") pod "network-check-target-4nvpb" (UID: "66284223-a2b7-4e40-978f-f44828b75e41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:03.501802 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.501689 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:04:02 +0000 UTC" deadline="2028-03-19 23:59:46.946384986 +0000 UTC" Jul 09 17:09:03.501802 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.501748 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14862h50m43.44464096s" Jul 09 17:09:03.665918 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.665859 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:03.666107 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.665872 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:03.666107 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.666007 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:03.666107 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:03.666054 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:03.676385 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.676264 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5x9qp" event={"ID":"b238ba72-80c9-42b3-a046-cc4cc546e62d","Type":"ContainerStarted","Data":"f83c4eba5ba73b1274b86cf26c971817e36a9b5eb20c4f40c80d1170b42ffa05"} Jul 09 17:09:03.678460 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.678351 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kshcf" event={"ID":"dbf9d27e-b21d-4436-8502-3543544a2fe1","Type":"ContainerStarted","Data":"d178be831fca45d193f052ef0b94733ef24502c8f419887109e90f17448264b7"} Jul 09 17:09:03.679640 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:03.679617 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l448f" event={"ID":"9204fadf-bf81-4ebb-bf8f-75b94fa2db53","Type":"ContainerStarted","Data":"fb01ad7b2f40f4291fad9497ad65bb6d9309b72f97ce1da473f4200f752f1475"} Jul 09 17:09:03.896662 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:03.896620 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c29139d_04e4_45ee_a125_a01247b1f99e.slice/crio-9a53eef68706ae661177be135f542164071af87d9c7f3e1bb0c5a021c998dd94 WatchSource:0}: Error finding container 9a53eef68706ae661177be135f542164071af87d9c7f3e1bb0c5a021c998dd94: Status 404 returned error can't find the container with id 9a53eef68706ae661177be135f542164071af87d9c7f3e1bb0c5a021c998dd94 Jul 09 17:09:03.896981 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:03.896950 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866a6276_35db_41bf_abc1_cdcf85a671cc.slice/crio-04ed5ae648de419599134589f605021ddceb1f0f4246d1f0fbdf80fc51c33cd0 WatchSource:0}: Error finding container 04ed5ae648de419599134589f605021ddceb1f0f4246d1f0fbdf80fc51c33cd0: Status 404 returned error can't find the container with id 04ed5ae648de419599134589f605021ddceb1f0f4246d1f0fbdf80fc51c33cd0 Jul 09 17:09:03.925719 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:03.925682 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58064e17_6c23_4e5f_991b_37c9969558c6.slice/crio-fb397073a4842f849b61b25a46b9c5e95bc02103f12c8812c2daf89b7246de76 WatchSource:0}: Error finding container fb397073a4842f849b61b25a46b9c5e95bc02103f12c8812c2daf89b7246de76: Status 404 returned error can't find the container with id fb397073a4842f849b61b25a46b9c5e95bc02103f12c8812c2daf89b7246de76 Jul 09 17:09:04.095256 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.095030 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:04.095429 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.095293 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:04.095429 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.095308 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:04.095429 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.095379 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret podName:c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:06.09535858 +0000 UTC m=+6.024804084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret") pod "global-pull-secret-syncer-txnlq" (UID: "c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:04.095626 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.095452 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:04.095626 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.095502 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs podName:ea9dd5df-3bc5-4eb7-a428-a93987879809 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:06.095486493 +0000 UTC m=+6.024931982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs") pod "network-metrics-daemon-hbwvp" (UID: "ea9dd5df-3bc5-4eb7-a428-a93987879809") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:04.196602 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.196567 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:04.197310 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.196789 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:09:04.197310 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.196810 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:09:04.197310 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.196823 2586 projected.go:196] Error preparing data for projected volume kube-api-access-tmddh for pod openshift-network-diagnostics/network-check-target-4nvpb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:04.197310 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.196880 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh podName:66284223-a2b7-4e40-978f-f44828b75e41 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:06.196862366 +0000 UTC m=+6.126307871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tmddh" (UniqueName: "kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh") pod "network-check-target-4nvpb" (UID: "66284223-a2b7-4e40-978f-f44828b75e41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:04.502266 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.502169 2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:04:02 +0000 UTC" deadline="2028-03-23 16:36:08.487447842 +0000 UTC" Jul 09 17:09:04.502266 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.502205 2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14951h27m3.9852465s" Jul 09 17:09:04.665988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.665954 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:04.666174 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:04.666098 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:04.684178 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.684124 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qwprr" event={"ID":"13e9a0cf-c2b6-4824-8447-ec09c276fd53","Type":"ContainerStarted","Data":"45905a0c851189f3e1c94d5e68a97714ff209b25d87a61ad826fa5287ad844f7"} Jul 09 17:09:04.685927 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.685898 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tklf4" event={"ID":"0c29139d-04e4-45ee-a125-a01247b1f99e","Type":"ContainerStarted","Data":"9a53eef68706ae661177be135f542164071af87d9c7f3e1bb0c5a021c998dd94"} Jul 09 17:09:04.689249 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.689218 2586 generic.go:356] "Generic (PLEG): container finished" podID="3aa7a67259a3462e5ba6775b001f3248" containerID="0a14b830c3c4135ce3598e220ddbfb97b1d10a1e83c04f4ce5cb873940a0e27b" exitCode=0 Jul 09 17:09:04.689394 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.689317 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" event={"ID":"3aa7a67259a3462e5ba6775b001f3248","Type":"ContainerDied","Data":"0a14b830c3c4135ce3598e220ddbfb97b1d10a1e83c04f4ce5cb873940a0e27b"} Jul 09 17:09:04.690938 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.690911 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" event={"ID":"58064e17-6c23-4e5f-991b-37c9969558c6","Type":"ContainerStarted","Data":"fb397073a4842f849b61b25a46b9c5e95bc02103f12c8812c2daf89b7246de76"} Jul 09 17:09:04.693632 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.693053 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d46qp" event={"ID":"35d602c3-6358-40e3-a13a-35f2a17c273d","Type":"ContainerStarted","Data":"587dae5a3ad95124daef3c68a9ac13b82e0a9af9546ac1a6401932071781b159"} Jul 09 17:09:04.700666 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.700545 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"b65589090ecabeaa88fcf67dd722cc5a784c2ef971d075e960534bb71e9727c7"} Jul 09 17:09:04.702503 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.702473 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" event={"ID":"866a6276-35db-41bf-abc1-cdcf85a671cc","Type":"ContainerStarted","Data":"04ed5ae648de419599134589f605021ddceb1f0f4246d1f0fbdf80fc51c33cd0"} Jul 09 17:09:04.704725 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.704700 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" event={"ID":"b45ab77f7adac80ae4f46c80d27c83b4","Type":"ContainerStarted","Data":"c20fe5973cbaec245212d09329bcb6e2aa9d4cb7972612a89c75747a6988f1c5"} Jul 09 17:09:04.727100 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:04.727035 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-186.ec2.internal" podStartSLOduration=3.72701148 podStartE2EDuration="3.72701148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:04.726600003 +0000 UTC m=+4.656045517" watchObservedRunningTime="2026-07-09 17:09:04.72701148 +0000 UTC m=+4.656457035" Jul 09 17:09:05.666307 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:05.666269 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:05.666808 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:05.666269 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:05.666808 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:05.666411 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:05.666808 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:05.666497 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:06.112343 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:06.111519 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:06.112343 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:06.111577 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:06.112343 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.111713 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:06.112343 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.111776 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs podName:ea9dd5df-3bc5-4eb7-a428-a93987879809 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:10.11175695 +0000 UTC m=+10.041202443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs") pod "network-metrics-daemon-hbwvp" (UID: "ea9dd5df-3bc5-4eb7-a428-a93987879809") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:06.112343 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.112203 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:06.112343 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.112252 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret podName:c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:10.112236603 +0000 UTC m=+10.041682096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret") pod "global-pull-secret-syncer-txnlq" (UID: "c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:06.212506 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:06.212369 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:06.212688 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.212589 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:09:06.212688 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.212612 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:09:06.212688 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.212626 2586 projected.go:196] Error preparing data for projected volume kube-api-access-tmddh for pod openshift-network-diagnostics/network-check-target-4nvpb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:06.212688 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.212689 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh podName:66284223-a2b7-4e40-978f-f44828b75e41 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:10.212669584 +0000 UTC m=+10.142115080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tmddh" (UniqueName: "kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh") pod "network-check-target-4nvpb" (UID: "66284223-a2b7-4e40-978f-f44828b75e41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:06.667556 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:06.667521 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:06.668099 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:06.667652 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:07.665737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:07.665706 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:07.665915 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:07.665839 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:07.666154 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:07.666127 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:07.666306 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:07.666283 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:08.665668 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:08.665615 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:08.666126 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:08.665761 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:09.666198 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.666109 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:09.666198 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.666166 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:09.666715 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:09.666249 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:09.666715 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:09.666627 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:09.717004 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.716965 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5x9qp" event={"ID":"b238ba72-80c9-42b3-a046-cc4cc546e62d","Type":"ContainerStarted","Data":"cde278fc2860b6d343aa5fd9905b5dfb7477e5f0a724542a2268e21a7dfb1dab"} Jul 09 17:09:09.719396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.719363 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l448f" event={"ID":"9204fadf-bf81-4ebb-bf8f-75b94fa2db53","Type":"ContainerStarted","Data":"b36802897ffca4df2b36c6550d562e5df2d32aba044a1d339fdfb5d806e9ce52"} Jul 09 17:09:09.722538 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.722509 2586 generic.go:356] "Generic (PLEG): container finished" podID="13e9a0cf-c2b6-4824-8447-ec09c276fd53" containerID="ebe85e8dd9034078e8d44b8544b683e4bb25263077024be0463b6d7828fe89e9" exitCode=0 Jul 09 17:09:09.722680 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.722602 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qwprr" event={"ID":"13e9a0cf-c2b6-4824-8447-ec09c276fd53","Type":"ContainerDied","Data":"ebe85e8dd9034078e8d44b8544b683e4bb25263077024be0463b6d7828fe89e9"} Jul 09 17:09:09.725489 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.725234 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" event={"ID":"3aa7a67259a3462e5ba6775b001f3248","Type":"ContainerStarted","Data":"6d8d8c3656868061e97c26c611dc9b68fbe555932da073f5da4556992411ad4e"} Jul 09 17:09:09.727956 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.727561 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kshcf" event={"ID":"dbf9d27e-b21d-4436-8502-3543544a2fe1","Type":"ContainerStarted","Data":"9f613f1a7bbe1fc0936f6449b81fcd3e2ccf482ea83ce1497a21ca4342ea79e2"} Jul 09 17:09:09.733191 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.733142 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5x9qp" podStartSLOduration=3.906753423 podStartE2EDuration="9.733125954s" totalImagesPullingTime="5.826372531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:09.732722611 +0000 UTC m=+9.662168124" watchObservedRunningTime="2026-07-09 17:09:09.733125954 +0000 UTC m=+9.662571468" Jul 09 17:09:09.766935 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.766853 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kshcf" podStartSLOduration=3.604757629 podStartE2EDuration="9.766833847s" totalImagesPullingTime="6.162076218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:09.7665674 +0000 UTC m=+9.696012913" watchObservedRunningTime="2026-07-09 17:09:09.766833847 +0000 UTC m=+9.696279359" Jul 09 17:09:09.767235 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.767155 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-186.ec2.internal" podStartSLOduration=8.767143264 podStartE2EDuration="8.767143264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:09.748679043 +0000 UTC m=+9.678124555" watchObservedRunningTime="2026-07-09 17:09:09.767143264 +0000 UTC m=+9.696588774" Jul 09 17:09:09.844167 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:09.844102 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l448f" podStartSLOduration=3.691694329 podStartE2EDuration="9.844076173s" totalImagesPullingTime="6.152381844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:09.790310362 +0000 UTC m=+9.719755899" watchObservedRunningTime="2026-07-09 17:09:09.844076173 +0000 UTC m=+9.773521685" Jul 09 17:09:10.153146 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:10.152355 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:10.153146 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:10.152417 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:10.153146 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.152588 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:10.153146 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.152660 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs podName:ea9dd5df-3bc5-4eb7-a428-a93987879809 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:18.152638772 +0000 UTC m=+18.082084278 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs") pod "network-metrics-daemon-hbwvp" (UID: "ea9dd5df-3bc5-4eb7-a428-a93987879809") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:10.153146 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.153057 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:10.153146 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.153109 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret podName:c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:18.153093605 +0000 UTC m=+18.082539098 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret") pod "global-pull-secret-syncer-txnlq" (UID: "c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:10.253025 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:10.252936 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:10.253198 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.253114 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:09:10.253198 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.253133 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:09:10.253198 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.253146 2586 projected.go:196] Error preparing data for projected volume kube-api-access-tmddh for pod openshift-network-diagnostics/network-check-target-4nvpb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:10.253349 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.253212 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh podName:66284223-a2b7-4e40-978f-f44828b75e41 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:18.253192939 +0000 UTC m=+18.182638441 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tmddh" (UniqueName: "kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh") pod "network-check-target-4nvpb" (UID: "66284223-a2b7-4e40-978f-f44828b75e41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:10.667762 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:10.667077 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:10.667762 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:10.667386 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:11.665393 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:11.665362 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:11.665707 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:11.665500 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:11.665707 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:11.665537 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:11.665707 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:11.665609 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:11.734098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:11.734058 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d46qp" event={"ID":"35d602c3-6358-40e3-a13a-35f2a17c273d","Type":"ContainerStarted","Data":"3d1f1ff2ca92f77c44fe69c2fd6a668bd293edef7f94c5291e7e2bab4390ade4"} Jul 09 17:09:11.752858 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:11.752794 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d46qp" podStartSLOduration=6.729404714 podStartE2EDuration="11.752772079s" totalImagesPullingTime="5.023367365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:11.751981671 +0000 UTC m=+11.681427184" watchObservedRunningTime="2026-07-09 17:09:11.752772079 +0000 UTC m=+11.682217594" Jul 09 17:09:12.563769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:12.563735 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:12.665577 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:12.665541 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:12.665735 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:12.665681 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:13.666098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:13.666051 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:13.667229 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:13.666186 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:13.667229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:13.666272 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:13.667229 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:13.666345 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:13.742036 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:13.741993 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" event={"ID":"866a6276-35db-41bf-abc1-cdcf85a671cc","Type":"ContainerStarted","Data":"01795ed444cf9e1592b890a5b7c625978740005d0e7ae2596547ad30491b0329"} Jul 09 17:09:13.744178 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:13.744144 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" event={"ID":"58064e17-6c23-4e5f-991b-37c9969558c6","Type":"ContainerStarted","Data":"4ad0c13a554f7d9c97da5219bf5a8bc8ed6609e9d6550992570331b7d8bd1fa2"} Jul 09 17:09:14.057732 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:14.057630 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:14.058387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:14.058328 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:14.075576 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:14.075517 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r6bmz" podStartSLOduration=5.26431436 podStartE2EDuration="14.075491742s" totalImagesPullingTime="8.811177382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:13.762601607 +0000 UTC m=+13.692047121" watchObservedRunningTime="2026-07-09 17:09:14.075491742 +0000 UTC m=+14.004937249" Jul 09 17:09:14.667205 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:14.666518 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:14.667205 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:14.666874 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:14.748407 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:14.748376 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l448f" Jul 09 17:09:15.665974 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:15.665937 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:15.666200 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:15.666072 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:15.666200 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:15.665941 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:15.666314 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:15.666237 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:16.665325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:16.665288 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:16.665709 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:16.665428 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:17.665510 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:17.665473 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:17.665510 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:17.665513 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:17.666038 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:17.665582 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:17.666038 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:17.665733 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:18.218500 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:18.218433 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:18.218768 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:18.218522 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:18.218768 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.218578 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:18.218768 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.218669 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret podName:c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:34.218647714 +0000 UTC m=+34.148093216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret") pod "global-pull-secret-syncer-txnlq" (UID: "c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:18.218768 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.218684 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:18.218768 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.218754 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs podName:ea9dd5df-3bc5-4eb7-a428-a93987879809 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:34.21873465 +0000 UTC m=+34.148180156 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs") pod "network-metrics-daemon-hbwvp" (UID: "ea9dd5df-3bc5-4eb7-a428-a93987879809") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:18.319884 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:18.319844 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:18.320065 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.320030 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:09:18.320065 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.320056 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:09:18.320171 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.320069 2586 projected.go:196] Error preparing data for projected volume kube-api-access-tmddh for pod openshift-network-diagnostics/network-check-target-4nvpb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:18.320171 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.320135 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh podName:66284223-a2b7-4e40-978f-f44828b75e41 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:34.320116116 +0000 UTC m=+34.249561624 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tmddh" (UniqueName: "kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh") pod "network-check-target-4nvpb" (UID: "66284223-a2b7-4e40-978f-f44828b75e41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:18.665513 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:18.665418 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:18.665665 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:18.665577 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:19.665454 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:19.665407 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:19.665639 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:19.665419 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:19.665639 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:19.665557 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:19.666044 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:19.665667 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:20.666677 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:20.666641 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:20.667145 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:20.666768 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:21.666065 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:21.666029 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:21.666252 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:21.666041 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:21.666252 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:21.666163 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:21.666360 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:21.666246 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:22.665358 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:22.665318 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:22.665810 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:22.665464 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:23.665652 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:23.665608 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:23.666191 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:23.665612 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:23.666191 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:23.665749 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:23.666191 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:23.665821 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:24.667391 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.667213 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:24.667391 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:24.667349 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:24.686993 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.686954 2586 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 17:09:24.774257 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.774224 2586 generic.go:356] "Generic (PLEG): container finished" podID="13e9a0cf-c2b6-4824-8447-ec09c276fd53" containerID="30137eee596ca6a2c15ba3d23c190fbcf9b902013b1b7d47d1f8b0366935e848" exitCode=0 Jul 09 17:09:24.774411 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.774304 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qwprr" event={"ID":"13e9a0cf-c2b6-4824-8447-ec09c276fd53","Type":"ContainerDied","Data":"30137eee596ca6a2c15ba3d23c190fbcf9b902013b1b7d47d1f8b0366935e848"} Jul 09 17:09:24.776075 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.775983 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tklf4" event={"ID":"0c29139d-04e4-45ee-a125-a01247b1f99e","Type":"ContainerStarted","Data":"cb3de572c1c7aae9484ceee513624a21dd3ab9003f456861244c48c0432de18f"} Jul 09 17:09:24.777552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.777535 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:09:24.777966 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.777941 2586 generic.go:356] "Generic (PLEG): container finished" podID="7cdfdc66-e339-48f1-bdb0-d930d999d0fc" containerID="1bc92ed00ac89e12d1776418178fbc0c3d88bce544c8db6780d6e56903c9acd5" exitCode=1 Jul 09 17:09:24.778095 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.778022 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerDied","Data":"1bc92ed00ac89e12d1776418178fbc0c3d88bce544c8db6780d6e56903c9acd5"} Jul 09 17:09:24.778154 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.778106 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"77d0fcafea97cc311197a288ee3f637c0f35365c85befe5576cff016333ae3a5"} Jul 09 17:09:24.780324 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.780232 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" event={"ID":"866a6276-35db-41bf-abc1-cdcf85a671cc","Type":"ContainerStarted","Data":"fe90868dcbf81c666fed2b3bcaa04a078720eeb9deb2504a1fbbdea657360e17"} Jul 09 17:09:24.818839 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:24.818712 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tklf4" podStartSLOduration=4.159116291 podStartE2EDuration="24.818690888s" totalImagesPullingTime="20.659574597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:24.818014503 +0000 UTC m=+24.747460033" watchObservedRunningTime="2026-07-09 17:09:24.818690888 +0000 UTC m=+24.748136401" Jul 09 17:09:25.519019 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.518720 2586 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T17:09:24.686989573Z","UUID":"5f8635c2-9f80-4666-a92f-a8e24ffceafc","Handler":null,"Name":"","Endpoint":""} Jul 09 17:09:25.520296 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.520275 2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 17:09:25.520491 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.520312 2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 17:09:25.665914 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.665877 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:25.666094 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.665877 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:25.666094 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:25.666003 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:25.666094 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:25.666072 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:25.785530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.785470 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:09:25.786111 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.785824 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"a1bf6dc6e1631e1165c33266a4fb23b70afcd26ca40d99cde3c0c3948b3dbb75"} Jul 09 17:09:25.786111 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.785891 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"71f609dff47ae627f36b769afc0f56a6d8732b950e16337ff412c57b16c98132"} Jul 09 17:09:25.786111 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.785927 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"d6ee4c862d0c1fd38f17a9b01e8258582f743db5c8428dd0980e2ac7b9cc24cb"} Jul 09 17:09:25.786111 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:25.785954 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"f184e61f6e466e11444e6da2ca8e1613f1edf9f6ff26d4205808a08fc0b5d140"} Jul 09 17:09:26.216585 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:26.216562 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kshcf_dbf9d27e-b21d-4436-8502-3543544a2fe1/dns-node-resolver/0.log" Jul 09 17:09:26.666008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:26.665975 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:26.666263 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:26.666081 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:26.790119 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:26.790086 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" event={"ID":"866a6276-35db-41bf-abc1-cdcf85a671cc","Type":"ContainerStarted","Data":"fefa53d4ff45112c3ca66ed68fc93a9275fc6eec646e330c82089492fbeb73a0"} Jul 09 17:09:26.792274 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:26.792248 2586 generic.go:356] "Generic (PLEG): container finished" podID="13e9a0cf-c2b6-4824-8447-ec09c276fd53" containerID="d403e5bd494dc4b5e2b3ea584ffc9bf48b3efcce8731a626cc974d5e11e33689" exitCode=0 Jul 09 17:09:26.792420 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:26.792283 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qwprr" event={"ID":"13e9a0cf-c2b6-4824-8447-ec09c276fd53","Type":"ContainerDied","Data":"d403e5bd494dc4b5e2b3ea584ffc9bf48b3efcce8731a626cc974d5e11e33689"} Jul 09 17:09:26.812252 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:26.812195 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dlr6x" podStartSLOduration=4.879316435 podStartE2EDuration="26.812178043s" totalImagesPullingTime="21.932861608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:26.811785868 +0000 UTC m=+26.741231399" watchObservedRunningTime="2026-07-09 17:09:26.812178043 +0000 UTC m=+26.741623554" Jul 09 17:09:26.992093 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:26.992070 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5x9qp_b238ba72-80c9-42b3-a046-cc4cc546e62d/node-ca/0.log" Jul 09 17:09:27.666068 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:27.665987 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:27.666271 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:27.666093 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:27.666271 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:27.666149 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:27.666352 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:27.666302 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:27.796986 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:27.796961 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:09:27.797402 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:27.797353 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"cd9ff71b8ea7d02254556ebb6a34ca36eecc48b218d4c00715cc04f914039c06"} Jul 09 17:09:28.666257 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:28.666224 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:28.666453 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:28.666347 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:28.802597 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:28.802568 2586 generic.go:356] "Generic (PLEG): container finished" podID="13e9a0cf-c2b6-4824-8447-ec09c276fd53" containerID="039055296df457ab6a9ad96c5334cc4cb8464d8f4de48871a6cc61e9f174406d" exitCode=0 Jul 09 17:09:28.802977 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:28.802609 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qwprr" event={"ID":"13e9a0cf-c2b6-4824-8447-ec09c276fd53","Type":"ContainerDied","Data":"039055296df457ab6a9ad96c5334cc4cb8464d8f4de48871a6cc61e9f174406d"} Jul 09 17:09:29.665484 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:29.665249 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:29.665484 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:29.665323 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:29.665722 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:29.665552 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:29.665722 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:29.665597 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:29.809301 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:29.809273 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:09:29.809756 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:29.809729 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"35218bca8d83d3c65d5c05b52ebcda0eb02e86a76a0f5b3eccf061ccd73ee44a"} Jul 09 17:09:29.810046 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:29.810021 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:29.810256 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:29.810240 2586 scope.go:117] "RemoveContainer" containerID="1bc92ed00ac89e12d1776418178fbc0c3d88bce544c8db6780d6e56903c9acd5" Jul 09 17:09:29.829737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:29.829714 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:30.666545 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:30.666509 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:30.666734 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:30.666600 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:30.818488 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:30.818461 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:09:30.818930 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:30.818770 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" event={"ID":"7cdfdc66-e339-48f1-bdb0-d930d999d0fc","Type":"ContainerStarted","Data":"f622ce85515997756ba986f760207b884faa4c0844965a14d9c62f9bd963fc2f"} Jul 09 17:09:30.819115 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:30.819094 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:30.819198 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:30.819187 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:30.835796 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:30.835772 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:09:30.854010 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:30.853949 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" podStartSLOduration=10.235166222 podStartE2EDuration="30.85392687s" totalImagesPullingTime="20.618760648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:30.853215881 +0000 UTC m=+30.782661396" watchObservedRunningTime="2026-07-09 17:09:30.85392687 +0000 UTC m=+30.783372383" Jul 09 17:09:31.665599 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:31.665566 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:31.665778 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:31.665571 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:31.665778 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:31.665667 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:31.665778 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:31.665755 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:32.666213 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:32.666177 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:32.666742 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:32.666315 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:33.666302 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:33.666263 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:33.666757 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:33.666313 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:33.666757 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:33.666415 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:33.666757 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:33.666530 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:34.230691 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:34.230648 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:34.230865 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:34.230735 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:34.230865 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.230837 2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:34.230983 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.230897 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret podName:c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4 nodeName:}" failed. No retries permitted until 2026-07-09 17:10:06.23087881 +0000 UTC m=+66.160324317 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret") pod "global-pull-secret-syncer-txnlq" (UID: "c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:34.230983 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.230837 2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:34.230983 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.230981 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs podName:ea9dd5df-3bc5-4eb7-a428-a93987879809 nodeName:}" failed. No retries permitted until 2026-07-09 17:10:06.230966194 +0000 UTC m=+66.160411689 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs") pod "network-metrics-daemon-hbwvp" (UID: "ea9dd5df-3bc5-4eb7-a428-a93987879809") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:34.331215 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:34.331178 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:34.331388 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.331361 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:09:34.331388 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.331380 2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:09:34.331388 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.331390 2586 projected.go:196] Error preparing data for projected volume kube-api-access-tmddh for pod openshift-network-diagnostics/network-check-target-4nvpb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:34.331551 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.331461 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh podName:66284223-a2b7-4e40-978f-f44828b75e41 nodeName:}" failed. No retries permitted until 2026-07-09 17:10:06.331429352 +0000 UTC m=+66.260874845 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tmddh" (UniqueName: "kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh") pod "network-check-target-4nvpb" (UID: "66284223-a2b7-4e40-978f-f44828b75e41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:34.665865 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:34.665787 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:34.665992 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:34.665921 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:35.665725 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:35.665689 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:35.666149 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:35.665785 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:35.666149 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:35.665845 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:35.666149 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:35.665962 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:35.832062 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:35.832029 2586 generic.go:356] "Generic (PLEG): container finished" podID="13e9a0cf-c2b6-4824-8447-ec09c276fd53" containerID="aa6afcb84449f1d38186d0fd87d611c102714298b6c148dd6a7c90992321996b" exitCode=0 Jul 09 17:09:35.832221 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:35.832075 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qwprr" event={"ID":"13e9a0cf-c2b6-4824-8447-ec09c276fd53","Type":"ContainerDied","Data":"aa6afcb84449f1d38186d0fd87d611c102714298b6c148dd6a7c90992321996b"} Jul 09 17:09:36.665998 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:36.665831 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:36.666388 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:36.666082 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:36.840106 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:36.840068 2586 generic.go:356] "Generic (PLEG): container finished" podID="13e9a0cf-c2b6-4824-8447-ec09c276fd53" containerID="829ecddb70e1aac1580b614ad13917d9319f116c4db02c547cb2e56c237f7522" exitCode=0 Jul 09 17:09:36.840240 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:36.840131 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qwprr" event={"ID":"13e9a0cf-c2b6-4824-8447-ec09c276fd53","Type":"ContainerDied","Data":"829ecddb70e1aac1580b614ad13917d9319f116c4db02c547cb2e56c237f7522"} Jul 09 17:09:37.665698 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:37.665665 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:37.665867 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:37.665670 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:37.665867 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:37.665776 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:37.665936 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:37.665870 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:37.846260 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:37.846230 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qwprr" event={"ID":"13e9a0cf-c2b6-4824-8447-ec09c276fd53","Type":"ContainerStarted","Data":"57b76c8875a16d6e5dc2f1ecc5548764ba90dbb1437af64989fe630a0c08355c"} Jul 09 17:09:37.870809 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:37.870761 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qwprr" podStartSLOduration=9.420791033 podStartE2EDuration="37.870746002s" totalImagesPullingTime="28.449954969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:37.868947975 +0000 UTC m=+37.798393504" watchObservedRunningTime="2026-07-09 17:09:37.870746002 +0000 UTC m=+37.800191513" Jul 09 17:09:38.666403 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:38.666363 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:38.666712 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:38.666522 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:39.666189 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:39.666155 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:39.666616 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:39.666279 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:39.666616 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:39.666337 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:39.666616 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:39.666453 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:40.666454 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:40.666406 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:40.666855 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:40.666515 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:41.666215 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:41.666182 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:41.666409 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:41.666222 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:41.666409 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:41.666284 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:41.666409 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:41.666403 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:42.666261 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:42.666217 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:42.666478 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:42.666364 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:43.666273 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:43.666224 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:43.666494 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:43.666336 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:43.666494 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:43.666395 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:43.666872 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:43.666531 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:44.665869 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:44.665836 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:44.666062 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:44.665947 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:45.665659 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:45.665619 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:45.666126 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:45.665744 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:45.666126 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:45.665789 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:45.666126 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:45.665879 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:46.665364 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:46.665335 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:46.665550 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:46.665503 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:47.495822 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:47.495790 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txnlq"] Jul 09 17:09:47.496541 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:47.495899 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:47.496541 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:47.496012 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:47.502937 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:47.502898 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4nvpb"] Jul 09 17:09:47.503259 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:47.503233 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:47.503403 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:47.503374 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:47.503752 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:47.503730 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hbwvp"] Jul 09 17:09:47.503849 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:47.503835 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:47.503972 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:47.503951 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:49.665735 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:49.665639 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:49.665735 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:49.665686 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:49.665735 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:49.665639 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:49.666323 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:49.665781 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hbwvp" podUID="ea9dd5df-3bc5-4eb7-a428-a93987879809" Jul 09 17:09:49.666323 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:49.665838 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-txnlq" podUID="c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4" Jul 09 17:09:49.666323 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:49.665896 2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4nvpb" podUID="66284223-a2b7-4e40-978f-f44828b75e41" Jul 09 17:09:51.407584 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.407279 2586 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-186.ec2.internal" event="NodeReady" Jul 09 17:09:51.407944 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.407691 2586 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 17:09:51.454035 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.454007 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jjlnk"] Jul 09 17:09:51.465855 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.465814 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m2h2b"] Jul 09 17:09:51.466010 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.465955 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.469473 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.469240 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 17:09:51.469473 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.469240 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 17:09:51.470004 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.469978 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kjrs9\"" Jul 09 17:09:51.479448 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.479413 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jjlnk"] Jul 09 17:09:51.479554 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.479476 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m2h2b"] Jul 09 17:09:51.479608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.479571 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.482625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.482599 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 17:09:51.482625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.482599 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 17:09:51.482815 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.482642 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ppxkf\"" Jul 09 17:09:51.482815 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.482642 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 17:09:51.482815 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.482744 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 17:09:51.552672 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552630 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-config-volume\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.552822 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552762 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da168133-4e14-41ca-a48b-590f0a5fc054-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.552822 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552796 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-metrics-tls\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.552822 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552814 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzbgd\" (UniqueName: \"kubernetes.io/projected/da168133-4e14-41ca-a48b-590f0a5fc054-kube-api-access-mzbgd\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.553001 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552836 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da168133-4e14-41ca-a48b-590f0a5fc054-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.553001 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552875 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-tmp-dir\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.553001 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552905 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znh9c\" (UniqueName: \"kubernetes.io/projected/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-kube-api-access-znh9c\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.553001 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552961 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da168133-4e14-41ca-a48b-590f0a5fc054-data-volume\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.553001 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.552989 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da168133-4e14-41ca-a48b-590f0a5fc054-crio-socket\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.560684 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.560659 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-f9xp9"] Jul 09 17:09:51.565596 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.565577 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f9xp9" Jul 09 17:09:51.568697 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.568675 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 17:09:51.568827 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.568715 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 17:09:51.568827 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.568722 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fb8qb\"" Jul 09 17:09:51.568827 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.568765 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 17:09:51.573742 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.573720 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f9xp9"] Jul 09 17:09:51.653503 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653468 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da168133-4e14-41ca-a48b-590f0a5fc054-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.653692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653510 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-metrics-tls\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.653692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653538 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzbgd\" (UniqueName: \"kubernetes.io/projected/da168133-4e14-41ca-a48b-590f0a5fc054-kube-api-access-mzbgd\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.653692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653592 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da168133-4e14-41ca-a48b-590f0a5fc054-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.653692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653618 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-tmp-dir\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.653692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653639 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znh9c\" (UniqueName: \"kubernetes.io/projected/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-kube-api-access-znh9c\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.653928 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653712 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jzc5\" (UniqueName: \"kubernetes.io/projected/bc2f9b71-3b62-4813-af2b-01a5efde8459-kube-api-access-8jzc5\") pod \"ingress-canary-f9xp9\" (UID: \"bc2f9b71-3b62-4813-af2b-01a5efde8459\") " pod="openshift-ingress-canary/ingress-canary-f9xp9" Jul 09 17:09:51.653928 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653756 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da168133-4e14-41ca-a48b-590f0a5fc054-data-volume\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.653928 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653789 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da168133-4e14-41ca-a48b-590f0a5fc054-crio-socket\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.653928 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653812 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc2f9b71-3b62-4813-af2b-01a5efde8459-cert\") pod \"ingress-canary-f9xp9\" (UID: \"bc2f9b71-3b62-4813-af2b-01a5efde8459\") " pod="openshift-ingress-canary/ingress-canary-f9xp9" Jul 09 17:09:51.653928 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.653843 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-config-volume\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.654137 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.654088 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-tmp-dir\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.654137 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.654121 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/da168133-4e14-41ca-a48b-590f0a5fc054-crio-socket\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.654238 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.654161 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/da168133-4e14-41ca-a48b-590f0a5fc054-data-volume\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.654315 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.654295 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/da168133-4e14-41ca-a48b-590f0a5fc054-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.654400 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.654384 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-config-volume\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.658068 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.657999 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/da168133-4e14-41ca-a48b-590f0a5fc054-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.658068 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.658014 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-metrics-tls\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.662372 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.662342 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzbgd\" (UniqueName: \"kubernetes.io/projected/da168133-4e14-41ca-a48b-590f0a5fc054-kube-api-access-mzbgd\") pod \"insights-runtime-extractor-m2h2b\" (UID: \"da168133-4e14-41ca-a48b-590f0a5fc054\") " pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.662482 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.662459 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znh9c\" (UniqueName: \"kubernetes.io/projected/0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e-kube-api-access-znh9c\") pod \"dns-default-jjlnk\" (UID: \"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e\") " pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.665354 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.665331 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:09:51.665462 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.665361 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:09:51.665517 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.665457 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:09:51.668325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.668267 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 17:09:51.668325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.668298 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w62rb\"" Jul 09 17:09:51.668549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.668347 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 17:09:51.668549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.668381 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x7mzk\"" Jul 09 17:09:51.668549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.668527 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 17:09:51.668684 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.668639 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 17:09:51.754919 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.754881 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzc5\" (UniqueName: \"kubernetes.io/projected/bc2f9b71-3b62-4813-af2b-01a5efde8459-kube-api-access-8jzc5\") pod \"ingress-canary-f9xp9\" (UID: \"bc2f9b71-3b62-4813-af2b-01a5efde8459\") " pod="openshift-ingress-canary/ingress-canary-f9xp9" Jul 09 17:09:51.755115 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.754941 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc2f9b71-3b62-4813-af2b-01a5efde8459-cert\") pod \"ingress-canary-f9xp9\" (UID: \"bc2f9b71-3b62-4813-af2b-01a5efde8459\") " pod="openshift-ingress-canary/ingress-canary-f9xp9" Jul 09 17:09:51.757275 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.757247 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bc2f9b71-3b62-4813-af2b-01a5efde8459-cert\") pod \"ingress-canary-f9xp9\" (UID: \"bc2f9b71-3b62-4813-af2b-01a5efde8459\") " pod="openshift-ingress-canary/ingress-canary-f9xp9" Jul 09 17:09:51.763891 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.763866 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jzc5\" (UniqueName: \"kubernetes.io/projected/bc2f9b71-3b62-4813-af2b-01a5efde8459-kube-api-access-8jzc5\") pod \"ingress-canary-f9xp9\" (UID: \"bc2f9b71-3b62-4813-af2b-01a5efde8459\") " pod="openshift-ingress-canary/ingress-canary-f9xp9" Jul 09 17:09:51.780746 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.780708 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:51.787808 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.787784 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m2h2b" Jul 09 17:09:51.874226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.874187 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-f9xp9" Jul 09 17:09:51.988559 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.988405 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m2h2b"] Jul 09 17:09:51.991727 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:51.991699 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jjlnk"] Jul 09 17:09:51.992109 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:51.992065 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda168133_4e14_41ca_a48b_590f0a5fc054.slice/crio-80a5cd65875394e2d8ed7f764d2e993f8908dc3a0a9353338a52f0152271051c WatchSource:0}: Error finding container 80a5cd65875394e2d8ed7f764d2e993f8908dc3a0a9353338a52f0152271051c: Status 404 returned error can't find the container with id 80a5cd65875394e2d8ed7f764d2e993f8908dc3a0a9353338a52f0152271051c Jul 09 17:09:51.994948 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:51.994927 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f4bab7d_8135_4bd5_a2ad_4e62f5fc727e.slice/crio-5a0da2ab1dfe0814cc5d098d2020b28a7c209f28e618cb98a9639cf094123fa1 WatchSource:0}: Error finding container 5a0da2ab1dfe0814cc5d098d2020b28a7c209f28e618cb98a9639cf094123fa1: Status 404 returned error can't find the container with id 5a0da2ab1dfe0814cc5d098d2020b28a7c209f28e618cb98a9639cf094123fa1 Jul 09 17:09:52.031918 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.031889 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-f9xp9"] Jul 09 17:09:52.035275 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:52.035249 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2f9b71_3b62_4813_af2b_01a5efde8459.slice/crio-af49775fea6d7ba553552b82de20cc5528cb84e04b39b50ce8ee259a70acd7db WatchSource:0}: Error finding container af49775fea6d7ba553552b82de20cc5528cb84e04b39b50ce8ee259a70acd7db: Status 404 returned error can't find the container with id af49775fea6d7ba553552b82de20cc5528cb84e04b39b50ce8ee259a70acd7db Jul 09 17:09:52.884803 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.884731 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f9xp9" event={"ID":"bc2f9b71-3b62-4813-af2b-01a5efde8459","Type":"ContainerStarted","Data":"af49775fea6d7ba553552b82de20cc5528cb84e04b39b50ce8ee259a70acd7db"} Jul 09 17:09:52.886229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.886145 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2h2b" event={"ID":"da168133-4e14-41ca-a48b-590f0a5fc054","Type":"ContainerStarted","Data":"80a5cd65875394e2d8ed7f764d2e993f8908dc3a0a9353338a52f0152271051c"} Jul 09 17:09:52.888263 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.888205 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjlnk" event={"ID":"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e","Type":"ContainerStarted","Data":"5a0da2ab1dfe0814cc5d098d2020b28a7c209f28e618cb98a9639cf094123fa1"} Jul 09 17:09:52.925668 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.925415 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc"] Jul 09 17:09:52.938026 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.937993 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc"] Jul 09 17:09:52.938195 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.938121 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" Jul 09 17:09:52.941566 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.941334 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 17:09:52.941566 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.941335 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hbmj8\"" Jul 09 17:09:52.967348 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:52.967309 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d7ad4a5-130a-469d-a834-49fa4ef6192b-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-f75lc\" (UID: \"6d7ad4a5-130a-469d-a834-49fa4ef6192b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" Jul 09 17:09:53.067977 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:53.067937 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d7ad4a5-130a-469d-a834-49fa4ef6192b-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-f75lc\" (UID: \"6d7ad4a5-130a-469d-a834-49fa4ef6192b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" Jul 09 17:09:53.068241 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:53.068217 2586 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 09 17:09:53.068310 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:09:53.068299 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d7ad4a5-130a-469d-a834-49fa4ef6192b-tls-certificates podName:6d7ad4a5-130a-469d-a834-49fa4ef6192b nodeName:}" failed. No retries permitted until 2026-07-09 17:09:53.568277461 +0000 UTC m=+53.497722957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/6d7ad4a5-130a-469d-a834-49fa4ef6192b-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-f75lc" (UID: "6d7ad4a5-130a-469d-a834-49fa4ef6192b") : secret "prometheus-operator-admission-webhook-tls" not found Jul 09 17:09:53.573416 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:53.573372 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d7ad4a5-130a-469d-a834-49fa4ef6192b-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-f75lc\" (UID: \"6d7ad4a5-130a-469d-a834-49fa4ef6192b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" Jul 09 17:09:53.579151 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:53.579114 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d7ad4a5-130a-469d-a834-49fa4ef6192b-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-f75lc\" (UID: \"6d7ad4a5-130a-469d-a834-49fa4ef6192b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" Jul 09 17:09:53.852129 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:53.852027 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" Jul 09 17:09:54.174473 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.174187 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-b9t6l"] Jul 09 17:09:54.203292 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.203251 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-b9t6l"] Jul 09 17:09:54.203483 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.203364 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-b9t6l" Jul 09 17:09:54.206815 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.206788 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 17:09:54.206928 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.206822 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fbpl6\"" Jul 09 17:09:54.207086 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.207067 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 17:09:54.279795 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.279769 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx4t7\" (UniqueName: \"kubernetes.io/projected/8e47f7c4-b0c9-4597-95c9-678ba1b3866d-kube-api-access-sx4t7\") pod \"downloads-68c9cbb497-b9t6l\" (UID: \"8e47f7c4-b0c9-4597-95c9-678ba1b3866d\") " pod="openshift-console/downloads-68c9cbb497-b9t6l" Jul 09 17:09:54.380733 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.380699 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx4t7\" (UniqueName: \"kubernetes.io/projected/8e47f7c4-b0c9-4597-95c9-678ba1b3866d-kube-api-access-sx4t7\") pod \"downloads-68c9cbb497-b9t6l\" (UID: \"8e47f7c4-b0c9-4597-95c9-678ba1b3866d\") " pod="openshift-console/downloads-68c9cbb497-b9t6l" Jul 09 17:09:54.390771 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.390743 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx4t7\" (UniqueName: \"kubernetes.io/projected/8e47f7c4-b0c9-4597-95c9-678ba1b3866d-kube-api-access-sx4t7\") pod \"downloads-68c9cbb497-b9t6l\" (UID: \"8e47f7c4-b0c9-4597-95c9-678ba1b3866d\") " pod="openshift-console/downloads-68c9cbb497-b9t6l" Jul 09 17:09:54.514976 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:54.514939 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-b9t6l" Jul 09 17:09:56.569627 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.569378 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc"] Jul 09 17:09:56.575254 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.575185 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-b9t6l"] Jul 09 17:09:56.582682 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:56.582645 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e47f7c4_b0c9_4597_95c9_678ba1b3866d.slice/crio-1bcbfc840a5d3c7259fb6373052a7e68009b85099ea5bcb45f2841045bedcff7 WatchSource:0}: Error finding container 1bcbfc840a5d3c7259fb6373052a7e68009b85099ea5bcb45f2841045bedcff7: Status 404 returned error can't find the container with id 1bcbfc840a5d3c7259fb6373052a7e68009b85099ea5bcb45f2841045bedcff7 Jul 09 17:09:56.583039 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:09:56.583003 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7ad4a5_130a_469d_a834_49fa4ef6192b.slice/crio-3cd560e054d46fbef4348e7559458fc8b3b83a5ab8f18f1155b76e30df5d6cb2 WatchSource:0}: Error finding container 3cd560e054d46fbef4348e7559458fc8b3b83a5ab8f18f1155b76e30df5d6cb2: Status 404 returned error can't find the container with id 3cd560e054d46fbef4348e7559458fc8b3b83a5ab8f18f1155b76e30df5d6cb2 Jul 09 17:09:56.903082 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.902991 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-b9t6l" event={"ID":"8e47f7c4-b0c9-4597-95c9-678ba1b3866d","Type":"ContainerStarted","Data":"1bcbfc840a5d3c7259fb6373052a7e68009b85099ea5bcb45f2841045bedcff7"} Jul 09 17:09:56.904477 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.904422 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2h2b" event={"ID":"da168133-4e14-41ca-a48b-590f0a5fc054","Type":"ContainerStarted","Data":"70bb5aeef608d02a5064e3bacd33e740ef27c0c60559951021639fdd7e1a0379"} Jul 09 17:09:56.906083 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.906059 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjlnk" event={"ID":"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e","Type":"ContainerStarted","Data":"462fa5d9c471492479edc8a26525dc388fbd2ce95105ce5a6e46f285ddf8c7fe"} Jul 09 17:09:56.906206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.906106 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jjlnk" event={"ID":"0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e","Type":"ContainerStarted","Data":"350f99b0bcb59d7c6fb90ceae442aaf7ace7247e3d1154d0fcfcf4799def205b"} Jul 09 17:09:56.906206 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.906157 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jjlnk" Jul 09 17:09:56.907259 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.907239 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" event={"ID":"6d7ad4a5-130a-469d-a834-49fa4ef6192b","Type":"ContainerStarted","Data":"3cd560e054d46fbef4348e7559458fc8b3b83a5ab8f18f1155b76e30df5d6cb2"} Jul 09 17:09:56.925157 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:56.925105 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jjlnk" podStartSLOduration=1.561939122 podStartE2EDuration="5.925087926s" totalImagesPullingTime="4.363148804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:56.924459296 +0000 UTC m=+56.853904824" watchObservedRunningTime="2026-07-09 17:09:56.925087926 +0000 UTC m=+56.854533437" Jul 09 17:09:57.912931 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:57.912891 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2h2b" event={"ID":"da168133-4e14-41ca-a48b-590f0a5fc054","Type":"ContainerStarted","Data":"bb97bd0e7440ee9da81f714635b517abc648a28fb8acffdcea3577b2770d3f9e"} Jul 09 17:09:58.919513 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:58.919468 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-f9xp9" event={"ID":"bc2f9b71-3b62-4813-af2b-01a5efde8459","Type":"ContainerStarted","Data":"92a1d4b290148e01744d401928412acfef778a0fecc67614679fd87fcbb198c6"} Jul 09 17:09:58.922142 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:58.922109 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" event={"ID":"6d7ad4a5-130a-469d-a834-49fa4ef6192b","Type":"ContainerStarted","Data":"6dd1f186a0f496904f2d5574c7f35c1a6201b2503a241efb8bf2b77c730cbb2e"} Jul 09 17:09:58.922473 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:58.922406 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" Jul 09 17:09:58.928957 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:58.928933 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" Jul 09 17:09:58.937364 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:58.937319 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-f9xp9" podStartSLOduration=1.573071265 podStartE2EDuration="7.937302204s" totalImagesPullingTime="6.364230939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:58.935810081 +0000 UTC m=+58.865255619" watchObservedRunningTime="2026-07-09 17:09:58.937302204 +0000 UTC m=+58.866747712" Jul 09 17:09:58.951985 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:58.951936 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-f75lc" podStartSLOduration=5.142771573 podStartE2EDuration="6.951920825s" totalImagesPullingTime="1.809149252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:58.95142468 +0000 UTC m=+58.880870211" watchObservedRunningTime="2026-07-09 17:09:58.951920825 +0000 UTC m=+58.881366336" Jul 09 17:09:59.991774 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:59.990725 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rj9v6"] Jul 09 17:09:59.996846 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:09:59.996820 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.002867 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.002371 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 09 17:10:00.002867 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.002386 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 17:10:00.002867 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.002698 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 09 17:10:00.002867 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.002714 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 17:10:00.002867 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.002810 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 17:10:00.003151 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.003066 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-slfmt\"" Jul 09 17:10:00.004406 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.004247 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rj9v6"] Jul 09 17:10:00.030307 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.030273 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84v2m\" (UniqueName: \"kubernetes.io/projected/eb64569e-f3cb-4b00-bba8-52ba45a65c90-kube-api-access-84v2m\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.030540 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.030367 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb64569e-f3cb-4b00-bba8-52ba45a65c90-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.030622 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.030598 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb64569e-f3cb-4b00-bba8-52ba45a65c90-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.030731 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.030682 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb64569e-f3cb-4b00-bba8-52ba45a65c90-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.131532 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.131492 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb64569e-f3cb-4b00-bba8-52ba45a65c90-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.131721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.131578 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb64569e-f3cb-4b00-bba8-52ba45a65c90-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.131721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.131643 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb64569e-f3cb-4b00-bba8-52ba45a65c90-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.131721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.131693 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84v2m\" (UniqueName: \"kubernetes.io/projected/eb64569e-f3cb-4b00-bba8-52ba45a65c90-kube-api-access-84v2m\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.132410 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.132379 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb64569e-f3cb-4b00-bba8-52ba45a65c90-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.134827 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.134798 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb64569e-f3cb-4b00-bba8-52ba45a65c90-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.135173 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.135145 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eb64569e-f3cb-4b00-bba8-52ba45a65c90-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.141169 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.141120 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84v2m\" (UniqueName: \"kubernetes.io/projected/eb64569e-f3cb-4b00-bba8-52ba45a65c90-kube-api-access-84v2m\") pod \"prometheus-operator-64785b6675-rj9v6\" (UID: \"eb64569e-f3cb-4b00-bba8-52ba45a65c90\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.309509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.309406 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" Jul 09 17:10:00.932366 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.932336 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rj9v6"] Jul 09 17:10:00.932882 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.932843 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2h2b" event={"ID":"da168133-4e14-41ca-a48b-590f0a5fc054","Type":"ContainerStarted","Data":"72d7958f2fe9c3843005863f440613683d733043ce1f01bdf1de99503215e5e8"} Jul 09 17:10:00.935625 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:00.935601 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb64569e_f3cb_4b00_bba8_52ba45a65c90.slice/crio-c73490ad75257f1791fdb650010b89bcab9b4c21cd8a9142872fb73390d929cd WatchSource:0}: Error finding container c73490ad75257f1791fdb650010b89bcab9b4c21cd8a9142872fb73390d929cd: Status 404 returned error can't find the container with id c73490ad75257f1791fdb650010b89bcab9b4c21cd8a9142872fb73390d929cd Jul 09 17:10:00.954315 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:00.954273 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m2h2b" podStartSLOduration=1.341715591 podStartE2EDuration="9.954258166s" totalImagesPullingTime="8.612542575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:00.953637979 +0000 UTC m=+60.883083509" watchObservedRunningTime="2026-07-09 17:10:00.954258166 +0000 UTC m=+60.883703655" Jul 09 17:10:01.937673 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:01.937632 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" event={"ID":"eb64569e-f3cb-4b00-bba8-52ba45a65c90","Type":"ContainerStarted","Data":"c73490ad75257f1791fdb650010b89bcab9b4c21cd8a9142872fb73390d929cd"} Jul 09 17:10:02.834961 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:02.834932 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcs4h" Jul 09 17:10:04.871767 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.871741 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f8b57b89-x8vj7"] Jul 09 17:10:04.873801 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.873776 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f8b57b89-x8vj7"] Jul 09 17:10:04.873928 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.873858 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:04.876990 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.876949 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 17:10:04.877141 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.876957 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 17:10:04.877141 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.877086 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 17:10:04.877244 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.877230 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2k2c9\"" Jul 09 17:10:04.877337 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.877319 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 17:10:04.877378 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.877339 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 17:10:04.950967 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.950928 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" event={"ID":"eb64569e-f3cb-4b00-bba8-52ba45a65c90","Type":"ContainerStarted","Data":"ba00682ebca9e09f61d053887cf3dc7f6ab5901617df95b9882aa8ea8b90ff7d"} Jul 09 17:10:04.951132 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.950986 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" event={"ID":"eb64569e-f3cb-4b00-bba8-52ba45a65c90","Type":"ContainerStarted","Data":"833c0ce266dcf70eb71c2ce4336efa5d7d6803712622de18e74f6f414961652c"} Jul 09 17:10:04.977750 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.977714 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-serving-cert\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:04.977950 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.977766 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-service-ca\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:04.977950 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.977864 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-oauth-serving-cert\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:04.978045 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.977971 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6j7m\" (UniqueName: \"kubernetes.io/projected/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-kube-api-access-t6j7m\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:04.978156 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.978126 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-config\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:04.978529 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:04.978483 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-oauth-config\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.079868 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.079821 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-oauth-config\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.079868 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.079870 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-serving-cert\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.080140 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.079904 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-service-ca\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.080140 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.079928 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-oauth-serving-cert\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.080140 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.079957 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6j7m\" (UniqueName: \"kubernetes.io/projected/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-kube-api-access-t6j7m\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.080140 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.079984 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-config\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.080753 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.080723 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-oauth-serving-cert\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.080899 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.080777 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-service-ca\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.080899 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.080835 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-config\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.082419 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.082393 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-oauth-config\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.082556 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.082539 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-serving-cert\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.090706 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.090685 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6j7m\" (UniqueName: \"kubernetes.io/projected/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-kube-api-access-t6j7m\") pod \"console-5f8b57b89-x8vj7\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.183113 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.183024 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:05.326661 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.326595 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-rj9v6" podStartSLOduration=3.052283567 podStartE2EDuration="6.326570312s" totalImagesPullingTime="3.274286745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:04.971483795 +0000 UTC m=+64.900929319" watchObservedRunningTime="2026-07-09 17:10:05.326570312 +0000 UTC m=+65.256015873" Jul 09 17:10:05.327252 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.327234 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f8b57b89-x8vj7"] Jul 09 17:10:05.329879 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:05.329846 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc1e7ce_792f_49c5_8c06_3058f6d25f0e.slice/crio-a3b53f3966b078a67bbb0b9af73bbe699d376369a79840fcd8b6407c1515a044 WatchSource:0}: Error finding container a3b53f3966b078a67bbb0b9af73bbe699d376369a79840fcd8b6407c1515a044: Status 404 returned error can't find the container with id a3b53f3966b078a67bbb0b9af73bbe699d376369a79840fcd8b6407c1515a044 Jul 09 17:10:05.955925 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:05.955882 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8b57b89-x8vj7" event={"ID":"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e","Type":"ContainerStarted","Data":"a3b53f3966b078a67bbb0b9af73bbe699d376369a79840fcd8b6407c1515a044"} Jul 09 17:10:06.290303 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.290242 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:10:06.290303 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.290308 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:10:06.293835 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.293808 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 17:10:06.293967 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.293883 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 17:10:06.303273 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.303242 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4-original-pull-secret\") pod \"global-pull-secret-syncer-txnlq\" (UID: \"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4\") " pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:10:06.303485 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.303254 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea9dd5df-3bc5-4eb7-a428-a93987879809-metrics-certs\") pod \"network-metrics-daemon-hbwvp\" (UID: \"ea9dd5df-3bc5-4eb7-a428-a93987879809\") " pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:10:06.385355 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.385315 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x7mzk\"" Jul 09 17:10:06.387354 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.387332 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txnlq" Jul 09 17:10:06.391202 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.391173 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:10:06.393140 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.393116 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hbwvp" Jul 09 17:10:06.394185 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.394169 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 17:10:06.404622 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.404553 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 17:10:06.414855 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.414820 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmddh\" (UniqueName: \"kubernetes.io/projected/66284223-a2b7-4e40-978f-f44828b75e41-kube-api-access-tmddh\") pod \"network-check-target-4nvpb\" (UID: \"66284223-a2b7-4e40-978f-f44828b75e41\") " pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:10:06.569316 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.569216 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txnlq"] Jul 09 17:10:06.573052 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:06.573017 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1a4c79c_6a26_4c67_8ae9_7ecad2fab6c4.slice/crio-e3bacd4f60bade7947d8f930a2eec31f9fdeb078150066d8e47a8c3181a8eee6 WatchSource:0}: Error finding container e3bacd4f60bade7947d8f930a2eec31f9fdeb078150066d8e47a8c3181a8eee6: Status 404 returned error can't find the container with id e3bacd4f60bade7947d8f930a2eec31f9fdeb078150066d8e47a8c3181a8eee6 Jul 09 17:10:06.595414 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.595385 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hbwvp"] Jul 09 17:10:06.599254 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:06.599221 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea9dd5df_3bc5_4eb7_a428_a93987879809.slice/crio-ed5f94286259260b2eac22319e1503824a3c83c51e53ad240411192194b096a5 WatchSource:0}: Error finding container ed5f94286259260b2eac22319e1503824a3c83c51e53ad240411192194b096a5: Status 404 returned error can't find the container with id ed5f94286259260b2eac22319e1503824a3c83c51e53ad240411192194b096a5 Jul 09 17:10:06.679906 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.679818 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w62rb\"" Jul 09 17:10:06.687671 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.687495 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:10:06.863055 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.862974 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4nvpb"] Jul 09 17:10:06.866088 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:06.866056 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66284223_a2b7_4e40_978f_f44828b75e41.slice/crio-c2966af9793cc513404f07161a1001a8f3b1765c21b44a2e1e29af20088e1e97 WatchSource:0}: Error finding container c2966af9793cc513404f07161a1001a8f3b1765c21b44a2e1e29af20088e1e97: Status 404 returned error can't find the container with id c2966af9793cc513404f07161a1001a8f3b1765c21b44a2e1e29af20088e1e97 Jul 09 17:10:06.915572 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.915543 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jjlnk" Jul 09 17:10:06.961502 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.961461 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txnlq" event={"ID":"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4","Type":"ContainerStarted","Data":"e3bacd4f60bade7947d8f930a2eec31f9fdeb078150066d8e47a8c3181a8eee6"} Jul 09 17:10:06.963565 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.963476 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4nvpb" event={"ID":"66284223-a2b7-4e40-978f-f44828b75e41","Type":"ContainerStarted","Data":"c2966af9793cc513404f07161a1001a8f3b1765c21b44a2e1e29af20088e1e97"} Jul 09 17:10:06.965282 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:06.965241 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hbwvp" event={"ID":"ea9dd5df-3bc5-4eb7-a428-a93987879809","Type":"ContainerStarted","Data":"ed5f94286259260b2eac22319e1503824a3c83c51e53ad240411192194b096a5"} Jul 09 17:10:07.347726 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.347659 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd"] Jul 09 17:10:07.371256 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.371209 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd"] Jul 09 17:10:07.372332 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.371305 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.373371 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.372970 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-hmsbr"] Jul 09 17:10:07.375561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.375248 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 09 17:10:07.375561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.375529 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 09 17:10:07.375961 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.375760 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-44knd\"" Jul 09 17:10:07.387003 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.386805 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.390893 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.390813 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-hmsbr"] Jul 09 17:10:07.391132 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.391107 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 09 17:10:07.391268 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.391232 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 09 17:10:07.391630 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.391459 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 09 17:10:07.391630 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.391544 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-zqssm\"" Jul 09 17:10:07.398798 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.398774 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ngcwp"] Jul 09 17:10:07.415251 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.415221 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.419487 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.419227 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 17:10:07.419487 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.419252 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7lwwq\"" Jul 09 17:10:07.419487 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.419266 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 17:10:07.419487 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.419324 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502626 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502689 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502729 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7257f390-a323-4bb0-889e-709297bcb63f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502766 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b401422-cf54-4944-96d1-2853d2f9b8f0-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502797 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502833 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwwjm\" (UniqueName: \"kubernetes.io/projected/02bfa02d-1348-4b6c-981e-798b801ba3b1-kube-api-access-zwwjm\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502862 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502892 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-tls\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502920 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7257f390-a323-4bb0-889e-709297bcb63f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502953 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-wtmp\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.502984 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02bfa02d-1348-4b6c-981e-798b801ba3b1-metrics-client-ca\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.503027 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-textfile\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.503058 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-root\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.503086 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.505845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.503118 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-sys\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.505845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.503150 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1b401422-cf54-4944-96d1-2853d2f9b8f0-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.505845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.503180 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7257f390-a323-4bb0-889e-709297bcb63f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.505845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.503207 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5t2f\" (UniqueName: \"kubernetes.io/projected/7257f390-a323-4bb0-889e-709297bcb63f-kube-api-access-h5t2f\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.505845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.503244 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx7bh\" (UniqueName: \"kubernetes.io/projected/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-api-access-cx7bh\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.604309 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604221 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx7bh\" (UniqueName: \"kubernetes.io/projected/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-api-access-cx7bh\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.604309 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604285 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.604561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604325 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604357 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7257f390-a323-4bb0-889e-709297bcb63f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.604561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604391 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b401422-cf54-4944-96d1-2853d2f9b8f0-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.604561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604417 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.604561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604463 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwwjm\" (UniqueName: \"kubernetes.io/projected/02bfa02d-1348-4b6c-981e-798b801ba3b1-kube-api-access-zwwjm\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604494 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604521 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-tls\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604561 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604543 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7257f390-a323-4bb0-889e-709297bcb63f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604570 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-wtmp\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604600 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02bfa02d-1348-4b6c-981e-798b801ba3b1-metrics-client-ca\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604635 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-textfile\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604660 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-root\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604682 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604708 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-sys\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604738 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1b401422-cf54-4944-96d1-2853d2f9b8f0-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604761 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7257f390-a323-4bb0-889e-709297bcb63f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.604978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.604786 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5t2f\" (UniqueName: \"kubernetes.io/projected/7257f390-a323-4bb0-889e-709297bcb63f-kube-api-access-h5t2f\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.605548 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.606037 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-accelerators-collector-config\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:10:07.606263 2586 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:10:07.606316 2586 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.606920 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02bfa02d-1348-4b6c-981e-798b801ba3b1-metrics-client-ca\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.607066 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b401422-cf54-4944-96d1-2853d2f9b8f0-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.607091 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-wtmp\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:10:07.606329 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-tls podName:1b401422-cf54-4944-96d1-2853d2f9b8f0 nodeName:}" failed. No retries permitted until 2026-07-09 17:10:08.106309377 +0000 UTC m=+68.035754869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-hmsbr" (UID: "1b401422-cf54-4944-96d1-2853d2f9b8f0") : secret "kube-state-metrics-tls" not found Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:10:07.607134 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-tls podName:02bfa02d-1348-4b6c-981e-798b801ba3b1 nodeName:}" failed. No retries permitted until 2026-07-09 17:10:08.107115931 +0000 UTC m=+68.036561424 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-tls") pod "node-exporter-ngcwp" (UID: "02bfa02d-1348-4b6c-981e-798b801ba3b1") : secret "node-exporter-tls" not found Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.607359 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1b401422-cf54-4944-96d1-2853d2f9b8f0-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.607403 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-root\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.607427 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02bfa02d-1348-4b6c-981e-798b801ba3b1-sys\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.607730 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.607690 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-textfile\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.608513 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.607926 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7257f390-a323-4bb0-889e-709297bcb63f-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.609918 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.609832 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.612485 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.612416 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7257f390-a323-4bb0-889e-709297bcb63f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.616588 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.616398 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.616588 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.616541 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx7bh\" (UniqueName: \"kubernetes.io/projected/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-api-access-cx7bh\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:07.620347 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.620242 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5t2f\" (UniqueName: \"kubernetes.io/projected/7257f390-a323-4bb0-889e-709297bcb63f-kube-api-access-h5t2f\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.622988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.622925 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwwjm\" (UniqueName: \"kubernetes.io/projected/02bfa02d-1348-4b6c-981e-798b801ba3b1-kube-api-access-zwwjm\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:07.624817 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.624758 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7257f390-a323-4bb0-889e-709297bcb63f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-nhbbd\" (UID: \"7257f390-a323-4bb0-889e-709297bcb63f\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.685851 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.685728 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" Jul 09 17:10:07.868433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.868351 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd"] Jul 09 17:10:07.871571 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:07.871335 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7257f390_a323_4bb0_889e_709297bcb63f.slice/crio-547d3b53833c37ca2a04d88c656a34bd6101138e8d8a7719a2551d06cc859325 WatchSource:0}: Error finding container 547d3b53833c37ca2a04d88c656a34bd6101138e8d8a7719a2551d06cc859325: Status 404 returned error can't find the container with id 547d3b53833c37ca2a04d88c656a34bd6101138e8d8a7719a2551d06cc859325 Jul 09 17:10:07.974336 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.974180 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" event={"ID":"7257f390-a323-4bb0-889e-709297bcb63f","Type":"ContainerStarted","Data":"8551be9e25bbc23e1bf313d5b63f30bb5398b12770a00eae71b8b102de66f57a"} Jul 09 17:10:07.974336 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:07.974253 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" event={"ID":"7257f390-a323-4bb0-889e-709297bcb63f","Type":"ContainerStarted","Data":"547d3b53833c37ca2a04d88c656a34bd6101138e8d8a7719a2551d06cc859325"} Jul 09 17:10:08.110846 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.110789 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:08.111038 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.110898 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-tls\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:08.114229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.114169 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b401422-cf54-4944-96d1-2853d2f9b8f0-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-hmsbr\" (UID: \"1b401422-cf54-4944-96d1-2853d2f9b8f0\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:08.114589 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.114432 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/02bfa02d-1348-4b6c-981e-798b801ba3b1-node-exporter-tls\") pod \"node-exporter-ngcwp\" (UID: \"02bfa02d-1348-4b6c-981e-798b801ba3b1\") " pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:08.305678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.305396 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" Jul 09 17:10:08.326738 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.326427 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ngcwp" Jul 09 17:10:08.449698 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.449663 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:10:08.453573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.453547 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.457012 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.456987 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 17:10:08.457213 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.457195 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 17:10:08.457283 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.457195 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 17:10:08.458461 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.457747 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 17:10:08.458461 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.457943 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 17:10:08.458461 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.457983 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 17:10:08.458461 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.458206 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7sd2g\"" Jul 09 17:10:08.458461 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.458216 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 17:10:08.458836 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.458533 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 17:10:08.458836 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.458715 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 17:10:08.467064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.467038 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:10:08.514836 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.514798 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.514836 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.514851 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515097 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.514885 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515097 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.514911 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515097 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.514936 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-config-out\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515097 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.514958 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515097 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.514986 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515097 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.515076 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-config-volume\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.515158 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.515186 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.515216 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.515247 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-web-config\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.515369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.515313 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh68j\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-kube-api-access-qh68j\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616484 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616545 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-web-config\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616616 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh68j\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-kube-api-access-qh68j\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616665 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616761 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616799 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616827 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616884 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-config-out\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616912 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.616942 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.617047 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-config-volume\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.617155 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.617184 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.618389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.617784 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.619403 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.618736 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.619403 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:10:08.619168 2586 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 17:10:08.619403 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:10:08.619228 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls podName:843c5e17-1dba-405e-a839-a108a442b27f nodeName:}" failed. No retries permitted until 2026-07-09 17:10:09.119207333 +0000 UTC m=+69.048652823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "843c5e17-1dba-405e-a839-a108a442b27f") : secret "alertmanager-main-tls" not found Jul 09 17:10:08.620821 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:10:08.620157 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle podName:843c5e17-1dba-405e-a839-a108a442b27f nodeName:}" failed. No retries permitted until 2026-07-09 17:10:09.120137595 +0000 UTC m=+69.049583099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "843c5e17-1dba-405e-a839-a108a442b27f") : configmap references non-existent config key: ca-bundle.crt Jul 09 17:10:08.623702 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.623644 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.627679 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.627630 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.628793 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.628717 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-config-volume\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.633153 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.633107 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh68j\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-kube-api-access-qh68j\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.640262 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.640203 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.640863 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.640812 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.641466 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.641405 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-web-config\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.642042 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.641997 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.646656 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.646613 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-config-out\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:08.655040 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:08.655006 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02bfa02d_1348_4b6c_981e_798b801ba3b1.slice/crio-0708d4a32de1d1bce6debcab398fbd969d0b20a8e84ad914fcd7d3dda896c655 WatchSource:0}: Error finding container 0708d4a32de1d1bce6debcab398fbd969d0b20a8e84ad914fcd7d3dda896c655: Status 404 returned error can't find the container with id 0708d4a32de1d1bce6debcab398fbd969d0b20a8e84ad914fcd7d3dda896c655 Jul 09 17:10:08.824387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.823726 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-hmsbr"] Jul 09 17:10:08.826597 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:08.826552 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b401422_cf54_4944_96d1_2853d2f9b8f0.slice/crio-9a8ad4f5df9f16e4fce77727ec26b11f5ab4356b16edee829b53ca14daa589eb WatchSource:0}: Error finding container 9a8ad4f5df9f16e4fce77727ec26b11f5ab4356b16edee829b53ca14daa589eb: Status 404 returned error can't find the container with id 9a8ad4f5df9f16e4fce77727ec26b11f5ab4356b16edee829b53ca14daa589eb Jul 09 17:10:08.985471 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.984602 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hbwvp" event={"ID":"ea9dd5df-3bc5-4eb7-a428-a93987879809","Type":"ContainerStarted","Data":"9d192e52ffbd97cdab449e5bc8a114f61310d70f098db942c66487b6adb438e8"} Jul 09 17:10:08.985471 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.984673 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hbwvp" event={"ID":"ea9dd5df-3bc5-4eb7-a428-a93987879809","Type":"ContainerStarted","Data":"d71e734d1de930b1cc1cd14f7ab076bb5e3bfe8c8de4cb0a135f8c152386dcb8"} Jul 09 17:10:08.990065 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.990019 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" event={"ID":"7257f390-a323-4bb0-889e-709297bcb63f","Type":"ContainerStarted","Data":"2cc65e5107807665f0dcb9cd2309591bf6e014ababd801deeb4dc2ae08f18daa"} Jul 09 17:10:08.992054 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.991976 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngcwp" event={"ID":"02bfa02d-1348-4b6c-981e-798b801ba3b1","Type":"ContainerStarted","Data":"0708d4a32de1d1bce6debcab398fbd969d0b20a8e84ad914fcd7d3dda896c655"} Jul 09 17:10:08.993706 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:08.993631 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" event={"ID":"1b401422-cf54-4944-96d1-2853d2f9b8f0","Type":"ContainerStarted","Data":"9a8ad4f5df9f16e4fce77727ec26b11f5ab4356b16edee829b53ca14daa589eb"} Jul 09 17:10:09.122650 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:09.122322 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:09.123486 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:09.123411 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:09.124691 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:09.124600 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:09.132970 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:09.132861 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:09.372548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:09.372244 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:10:10.441991 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.441909 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hbwvp" podStartSLOduration=68.381311821 podStartE2EDuration="1m10.441883162s" totalImagesPullingTime="2.060571341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:09.003179377 +0000 UTC m=+68.932624890" watchObservedRunningTime="2026-07-09 17:10:10.441883162 +0000 UTC m=+70.371328676" Jul 09 17:10:10.443647 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.443619 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg"] Jul 09 17:10:10.452897 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.452860 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.456052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.456023 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 09 17:10:10.456207 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.456081 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-eo76qoo2actnj\"" Jul 09 17:10:10.456207 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.456023 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dj8dr\"" Jul 09 17:10:10.456342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.456319 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 09 17:10:10.456470 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.456433 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 09 17:10:10.456551 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.456491 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 09 17:10:10.457388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.457367 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 09 17:10:10.461360 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.461335 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg"] Jul 09 17:10:10.538605 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.538574 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.538806 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.538626 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-grpc-tls\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.538806 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.538708 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-tls\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.538806 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.538743 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.538806 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.538792 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83b49f60-733f-4618-893a-e2b2f61969f1-metrics-client-ca\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.539022 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.538895 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgxmq\" (UniqueName: \"kubernetes.io/projected/83b49f60-733f-4618-893a-e2b2f61969f1-kube-api-access-zgxmq\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.539022 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.538954 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.539022 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.539001 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.639421 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.639317 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.639421 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.639357 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.639421 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.639383 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-grpc-tls\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.639421 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.639419 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-tls\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.640015 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.639988 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.640090 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.640039 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83b49f60-733f-4618-893a-e2b2f61969f1-metrics-client-ca\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.640139 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.640086 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgxmq\" (UniqueName: \"kubernetes.io/projected/83b49f60-733f-4618-893a-e2b2f61969f1-kube-api-access-zgxmq\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.640187 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.640146 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.641613 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.641216 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83b49f60-733f-4618-893a-e2b2f61969f1-metrics-client-ca\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.645082 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.645048 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.645220 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.645061 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.645932 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.645888 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.646867 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.646812 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-tls\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.649527 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.649503 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-grpc-tls\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.649944 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.649923 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/83b49f60-733f-4618-893a-e2b2f61969f1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.651016 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.650973 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgxmq\" (UniqueName: \"kubernetes.io/projected/83b49f60-733f-4618-893a-e2b2f61969f1-kube-api-access-zgxmq\") pod \"thanos-querier-c8d57cbb5-s7vtg\" (UID: \"83b49f60-733f-4618-893a-e2b2f61969f1\") " pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:10.768543 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:10.768506 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:11.778228 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.778191 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-77597554fc-57cgb"] Jul 09 17:10:11.780570 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.780547 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.783734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.783703 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 09 17:10:11.783734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.783725 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-d7pc7\"" Jul 09 17:10:11.783894 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.783744 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 09 17:10:11.783894 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.783806 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 17:10:11.785568 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.785547 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 09 17:10:11.785689 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.785584 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ff65kdorl4d08\"" Jul 09 17:10:11.794834 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.794806 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77597554fc-57cgb"] Jul 09 17:10:11.850017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.849982 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-client-ca-bundle\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.850197 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.850054 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9f41f1d0-2a3c-446f-aaff-17490f9017a8-metrics-server-audit-profiles\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.850197 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.850130 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9f41f1d0-2a3c-446f-aaff-17490f9017a8-audit-log\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.850197 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.850172 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r4qd\" (UniqueName: \"kubernetes.io/projected/9f41f1d0-2a3c-446f-aaff-17490f9017a8-kube-api-access-5r4qd\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.850343 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.850256 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f41f1d0-2a3c-446f-aaff-17490f9017a8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.850343 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.850324 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-secret-metrics-server-client-certs\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.850454 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.850353 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-secret-metrics-server-tls\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.951559 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.951513 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9f41f1d0-2a3c-446f-aaff-17490f9017a8-metrics-server-audit-profiles\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.951754 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.951584 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9f41f1d0-2a3c-446f-aaff-17490f9017a8-audit-log\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.951754 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.951614 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r4qd\" (UniqueName: \"kubernetes.io/projected/9f41f1d0-2a3c-446f-aaff-17490f9017a8-kube-api-access-5r4qd\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.951754 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.951649 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f41f1d0-2a3c-446f-aaff-17490f9017a8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.951754 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.951721 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-secret-metrics-server-client-certs\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.951754 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.951746 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-secret-metrics-server-tls\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.952052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.951776 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-client-ca-bundle\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.952103 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.952063 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9f41f1d0-2a3c-446f-aaff-17490f9017a8-audit-log\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.952521 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.952473 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f41f1d0-2a3c-446f-aaff-17490f9017a8-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.952719 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.952694 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9f41f1d0-2a3c-446f-aaff-17490f9017a8-metrics-server-audit-profiles\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.954139 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.954110 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-secret-metrics-server-client-certs\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.954318 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.954298 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-secret-metrics-server-tls\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.954393 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.954305 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f41f1d0-2a3c-446f-aaff-17490f9017a8-client-ca-bundle\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:11.960697 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:11.960672 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r4qd\" (UniqueName: \"kubernetes.io/projected/9f41f1d0-2a3c-446f-aaff-17490f9017a8-kube-api-access-5r4qd\") pod \"metrics-server-77597554fc-57cgb\" (UID: \"9f41f1d0-2a3c-446f-aaff-17490f9017a8\") " pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:12.092707 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.092613 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:12.143631 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.143596 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns"] Jul 09 17:10:12.145992 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.145910 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:12.149529 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.149504 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rbwwt\"" Jul 09 17:10:12.150147 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.150115 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 17:10:12.160967 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.160938 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns"] Jul 09 17:10:12.255975 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.255932 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mftp2\" (UniqueName: \"kubernetes.io/projected/422fa30b-b3e1-42e8-9c32-4a34b97f53cd-kube-api-access-mftp2\") pod \"monitoring-plugin-5f68df7f58-dtzns\" (UID: \"422fa30b-b3e1-42e8-9c32-4a34b97f53cd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:12.256196 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.255994 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/422fa30b-b3e1-42e8-9c32-4a34b97f53cd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-dtzns\" (UID: \"422fa30b-b3e1-42e8-9c32-4a34b97f53cd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:12.357323 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.357219 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mftp2\" (UniqueName: \"kubernetes.io/projected/422fa30b-b3e1-42e8-9c32-4a34b97f53cd-kube-api-access-mftp2\") pod \"monitoring-plugin-5f68df7f58-dtzns\" (UID: \"422fa30b-b3e1-42e8-9c32-4a34b97f53cd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:12.357323 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.357272 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/422fa30b-b3e1-42e8-9c32-4a34b97f53cd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-dtzns\" (UID: \"422fa30b-b3e1-42e8-9c32-4a34b97f53cd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:12.359980 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.359951 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/422fa30b-b3e1-42e8-9c32-4a34b97f53cd-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-dtzns\" (UID: \"422fa30b-b3e1-42e8-9c32-4a34b97f53cd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:12.366897 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.366870 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mftp2\" (UniqueName: \"kubernetes.io/projected/422fa30b-b3e1-42e8-9c32-4a34b97f53cd-kube-api-access-mftp2\") pod \"monitoring-plugin-5f68df7f58-dtzns\" (UID: \"422fa30b-b3e1-42e8-9c32-4a34b97f53cd\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:12.460299 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.460194 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:12.652773 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.652671 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw"] Jul 09 17:10:12.655531 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.655502 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.658570 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.658544 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zm44s\"" Jul 09 17:10:12.658721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.658696 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 09 17:10:12.658786 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.658763 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 09 17:10:12.659042 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.658989 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 09 17:10:12.659180 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.658993 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 09 17:10:12.659285 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.659195 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 09 17:10:12.665291 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.665270 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 09 17:10:12.671333 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.671314 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw"] Jul 09 17:10:12.761339 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.761297 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn8qg\" (UniqueName: \"kubernetes.io/projected/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-kube-api-access-cn8qg\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.761573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.761351 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.761573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.761382 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-federate-client-tls\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.761573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.761418 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-serving-certs-ca-bundle\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.761573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.761472 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-secret-telemeter-client\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.761573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.761524 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-telemeter-client-tls\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.761800 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.761601 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-metrics-client-ca\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.761800 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.761693 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.862402 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.862361 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.862904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.862451 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn8qg\" (UniqueName: \"kubernetes.io/projected/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-kube-api-access-cn8qg\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.862904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.862488 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.862904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.862519 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-federate-client-tls\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.862904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.862561 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-serving-certs-ca-bundle\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.862904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.862599 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-secret-telemeter-client\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.862904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.862651 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-telemeter-client-tls\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.862904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.862691 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-metrics-client-ca\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.863474 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.863395 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.863603 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.863588 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-serving-certs-ca-bundle\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.863680 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.863615 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-metrics-client-ca\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.865702 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.865655 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-secret-telemeter-client\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.865828 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.865733 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.866034 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.866009 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-federate-client-tls\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.866150 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.866052 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-telemeter-client-tls\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.872045 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.872023 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn8qg\" (UniqueName: \"kubernetes.io/projected/fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8-kube-api-access-cn8qg\") pod \"telemeter-client-6f79d4fbc9-pmfsw\" (UID: \"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8\") " pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:12.969321 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:12.969285 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" Jul 09 17:10:13.631614 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.631566 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:10:13.635562 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.635538 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.638713 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.638664 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 17:10:13.639116 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.638991 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 17:10:13.639252 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.639212 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7cli00m0kjqqp\"" Jul 09 17:10:13.639252 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.639243 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 17:10:13.639499 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.639308 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 17:10:13.639499 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.639360 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 17:10:13.639499 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.639384 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 17:10:13.639860 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.639791 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 17:10:13.641236 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.641214 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 17:10:13.641377 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.641359 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 17:10:13.641513 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.641497 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 17:10:13.641594 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.641578 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 17:10:13.641799 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.641782 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 17:10:13.642001 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.641985 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-94drg\"" Jul 09 17:10:13.642052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.642007 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 17:10:13.646239 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.646219 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 17:10:13.653100 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.653072 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:10:13.774027 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.773981 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-config\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774027 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774031 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-web-config\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774278 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774066 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774278 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774094 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774278 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774120 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774278 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774143 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774278 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774258 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774525 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774290 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774525 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774325 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774525 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774389 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774525 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774435 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774525 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774490 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774726 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774526 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-config-out\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774726 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774551 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsc87\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-kube-api-access-qsc87\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774726 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774586 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774726 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774607 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774726 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774669 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774945 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774751 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774945 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774779 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.774945 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.774804 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.875963 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.875924 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.875963 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.875970 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876008 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876035 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876065 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876096 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876139 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-config-out\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876163 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsc87\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-kube-api-access-qsc87\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876204 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876227 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876253 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876279 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876304 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876329 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876372 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-config\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.876552 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876394 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-web-config\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.877229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876428 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.877229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876485 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.877229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876508 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.877229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876533 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.877229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876817 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.877229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.876943 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.877537 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.877301 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.879799 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.879422 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.879799 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.879618 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.879982 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.879943 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.880223 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.880199 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.880809 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.880778 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.881138 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.881113 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.885713 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.885651 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.886304 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.886263 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-web-config\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.886413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.886364 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.886497 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.886412 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.887762 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.887630 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-config-out\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.887895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.887874 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-config\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.887962 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.887893 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.888285 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.888262 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.888583 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.888540 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsc87\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-kube-api-access-qsc87\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.888670 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.888628 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.888805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.888787 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:13.950185 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:13.950142 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:22.061331 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.057588 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg"] Jul 09 17:10:22.063761 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:22.063608 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b49f60_733f_4618_893a_e2b2f61969f1.slice/crio-dc7526ca9a2bbce0da0877481f976319959b9a29c063d9598a9a1ed3a69f4f70 WatchSource:0}: Error finding container dc7526ca9a2bbce0da0877481f976319959b9a29c063d9598a9a1ed3a69f4f70: Status 404 returned error can't find the container with id dc7526ca9a2bbce0da0877481f976319959b9a29c063d9598a9a1ed3a69f4f70 Jul 09 17:10:22.127078 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.126942 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8b57b89-x8vj7" event={"ID":"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e","Type":"ContainerStarted","Data":"801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604"} Jul 09 17:10:22.137301 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.136968 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-b9t6l" Jul 09 17:10:22.141289 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.141261 2586 patch_prober.go:28] interesting pod/downloads-68c9cbb497-b9t6l container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused" start-of-body= Jul 09 17:10:22.141507 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.141473 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-b9t6l" podUID="8e47f7c4-b0c9-4597-95c9-678ba1b3866d" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.10:8080/\": dial tcp 10.134.0.10:8080: connect: connection refused" Jul 09 17:10:22.144263 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.142307 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:10:22.145602 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.145578 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngcwp" event={"ID":"02bfa02d-1348-4b6c-981e-798b801ba3b1","Type":"ContainerStarted","Data":"4f5674a67e41917ea2814c9556110666cba722e211b4d812fcb32ebf533fd691"} Jul 09 17:10:22.150131 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.150086 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f8b57b89-x8vj7" podStartSLOduration=2.067366012 podStartE2EDuration="18.150070923s" totalImagesPullingTime="16.082704911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:22.148730459 +0000 UTC m=+82.078175973" watchObservedRunningTime="2026-07-09 17:10:22.150070923 +0000 UTC m=+82.079516439" Jul 09 17:10:22.158857 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.158831 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:10:22.186722 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.186464 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4nvpb" podStartSLOduration=67.273254833 podStartE2EDuration="1m22.186425347s" totalImagesPullingTime="14.913170514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:22.175944939 +0000 UTC m=+82.105390451" watchObservedRunningTime="2026-07-09 17:10:22.186425347 +0000 UTC m=+82.115870855" Jul 09 17:10:22.218751 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.213604 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw"] Jul 09 17:10:22.231389 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.230063 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-b9t6l" podStartSLOduration=3.030805828 podStartE2EDuration="28.230043039s" totalImagesPullingTime="25.199237211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:22.226825839 +0000 UTC m=+82.156271351" watchObservedRunningTime="2026-07-09 17:10:22.230043039 +0000 UTC m=+82.159488552" Jul 09 17:10:22.241583 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.238888 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns"] Jul 09 17:10:22.246721 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:22.246629 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422fa30b_b3e1_42e8_9c32_4a34b97f53cd.slice/crio-928ac0953c4dc91d138ef9d343a6e96b9909df490812e8016dd9d0329bebdd27 WatchSource:0}: Error finding container 928ac0953c4dc91d138ef9d343a6e96b9909df490812e8016dd9d0329bebdd27: Status 404 returned error can't find the container with id 928ac0953c4dc91d138ef9d343a6e96b9909df490812e8016dd9d0329bebdd27 Jul 09 17:10:22.374405 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.374297 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77597554fc-57cgb"] Jul 09 17:10:22.438565 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:22.437769 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:10:22.443471 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:10:22.443340 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843c5e17_1dba_405e_a839_a108a442b27f.slice/crio-ae08c6f910f370c8f9d48ee80935ed802b724bae8af8e20ee0d3e4cef3b71840 WatchSource:0}: Error finding container ae08c6f910f370c8f9d48ee80935ed802b724bae8af8e20ee0d3e4cef3b71840: Status 404 returned error can't find the container with id ae08c6f910f370c8f9d48ee80935ed802b724bae8af8e20ee0d3e4cef3b71840 Jul 09 17:10:23.161563 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.161516 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" event={"ID":"7257f390-a323-4bb0-889e-709297bcb63f","Type":"ContainerStarted","Data":"20a8966b2c2d5c9c127cfd4e7ed42c6cec9b3992751ead4711376eff1e338d11"} Jul 09 17:10:23.164988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.164909 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" event={"ID":"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8","Type":"ContainerStarted","Data":"68fd96176c5176864895e5c1d60b9fe5cce57f3f50a0aa7c563ee4d4f8bd8254"} Jul 09 17:10:23.169731 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.168876 2586 generic.go:356] "Generic (PLEG): container finished" podID="02bfa02d-1348-4b6c-981e-798b801ba3b1" containerID="4f5674a67e41917ea2814c9556110666cba722e211b4d812fcb32ebf533fd691" exitCode=0 Jul 09 17:10:23.169731 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.168957 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngcwp" event={"ID":"02bfa02d-1348-4b6c-981e-798b801ba3b1","Type":"ContainerDied","Data":"4f5674a67e41917ea2814c9556110666cba722e211b4d812fcb32ebf533fd691"} Jul 09 17:10:23.174629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.174428 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" event={"ID":"83b49f60-733f-4618-893a-e2b2f61969f1","Type":"ContainerStarted","Data":"dc7526ca9a2bbce0da0877481f976319959b9a29c063d9598a9a1ed3a69f4f70"} Jul 09 17:10:23.178376 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.178280 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerStarted","Data":"ae08c6f910f370c8f9d48ee80935ed802b724bae8af8e20ee0d3e4cef3b71840"} Jul 09 17:10:23.186155 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.186048 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" event={"ID":"9f41f1d0-2a3c-446f-aaff-17490f9017a8","Type":"ContainerStarted","Data":"64606abae0740a40733388823687e0ace12b4bb0a4f9b266e4d71512b721ffe8"} Jul 09 17:10:23.190223 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.190184 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-b9t6l" event={"ID":"8e47f7c4-b0c9-4597-95c9-678ba1b3866d","Type":"ContainerStarted","Data":"38f997bc8c70db5202c97f25bebbc5c01fd0353b26a83bdafc5feecfa077248e"} Jul 09 17:10:23.194577 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.194549 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txnlq" event={"ID":"c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4","Type":"ContainerStarted","Data":"5b7f10354fbf43b783f3eae7e9d74af4cb10f612bd4879fa0110cde2bc73b3f2"} Jul 09 17:10:23.196299 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.196231 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" event={"ID":"422fa30b-b3e1-42e8-9c32-4a34b97f53cd","Type":"ContainerStarted","Data":"928ac0953c4dc91d138ef9d343a6e96b9909df490812e8016dd9d0329bebdd27"} Jul 09 17:10:23.200113 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.199052 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerStarted","Data":"2b96cf6defbb3259ae7f65f05ec83abf759571593ca41b9342f24531fa2ffc35"} Jul 09 17:10:23.203732 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.203658 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4nvpb" event={"ID":"66284223-a2b7-4e40-978f-f44828b75e41","Type":"ContainerStarted","Data":"8e3e9fe57fc49bed26779a43ff05f8dbeab4010b821ce1046bd75a0a6b9f8218"} Jul 09 17:10:23.206148 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.204974 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-nhbbd" podStartSLOduration=2.463841647 podStartE2EDuration="16.204956748s" totalImagesPullingTime="13.741115101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:23.18297319 +0000 UTC m=+83.112418703" watchObservedRunningTime="2026-07-09 17:10:23.204956748 +0000 UTC m=+83.134402259" Jul 09 17:10:23.210327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.209799 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" event={"ID":"1b401422-cf54-4944-96d1-2853d2f9b8f0","Type":"ContainerStarted","Data":"604149de864c73ea6379e1c800f36ecdfcc66d5952973b00f97e80baa86b3297"} Jul 09 17:10:23.210327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.209852 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" event={"ID":"1b401422-cf54-4944-96d1-2853d2f9b8f0","Type":"ContainerStarted","Data":"08d29c87023e936918d1d2f080ff7e5d5ba0bfc4bbd616908803372337e23f16"} Jul 09 17:10:23.210327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.209882 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" event={"ID":"1b401422-cf54-4944-96d1-2853d2f9b8f0","Type":"ContainerStarted","Data":"4a634b503776480b77151a0acd54543e12dfa629573f6384283c7e18e47f2e21"} Jul 09 17:10:23.222451 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.222363 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-b9t6l" Jul 09 17:10:23.229652 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:23.228186 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-txnlq" podStartSLOduration=67.932812794 podStartE2EDuration="1m23.228167339s" totalImagesPullingTime="15.295354545s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:23.227669366 +0000 UTC m=+83.157114899" watchObservedRunningTime="2026-07-09 17:10:23.228167339 +0000 UTC m=+83.157612852" Jul 09 17:10:24.223006 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:24.222821 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngcwp" event={"ID":"02bfa02d-1348-4b6c-981e-798b801ba3b1","Type":"ContainerStarted","Data":"54304d54ad8286d66d524b2fe1c2ae42dff520f49420e835d42025bc97ba34e0"} Jul 09 17:10:24.223006 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:24.222922 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngcwp" event={"ID":"02bfa02d-1348-4b6c-981e-798b801ba3b1","Type":"ContainerStarted","Data":"0d80d6d05e6ed1ebf8f7a60c38057f7a18f4e1cca248c1b211d00930e9410dfa"} Jul 09 17:10:24.247307 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:24.246583 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-hmsbr" podStartSLOduration=4.277965229 podStartE2EDuration="17.246548067s" totalImagesPullingTime="12.968582838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:23.283502817 +0000 UTC m=+83.212948344" watchObservedRunningTime="2026-07-09 17:10:24.246548067 +0000 UTC m=+84.175993579" Jul 09 17:10:24.677520 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:24.675971 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ngcwp" podStartSLOduration=4.729079363 podStartE2EDuration="17.675945491s" totalImagesPullingTime="12.758688459s" totalInitContainerRuntime="188.177669ms" isStatefulPod=true podCreationTimestamp="2026-07-09 17:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:24.24728932 +0000 UTC m=+84.176734832" watchObservedRunningTime="2026-07-09 17:10:24.675945491 +0000 UTC m=+84.605391005" Jul 09 17:10:24.677520 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:24.676848 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f8b57b89-x8vj7"] Jul 09 17:10:25.184298 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:25.184205 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:29.256588 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.256285 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" event={"ID":"9f41f1d0-2a3c-446f-aaff-17490f9017a8","Type":"ContainerStarted","Data":"66438616024d91f19d3e5870028d242622b18b73c5a1451baf74bec06efb36df"} Jul 09 17:10:29.258292 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.258261 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" event={"ID":"422fa30b-b3e1-42e8-9c32-4a34b97f53cd","Type":"ContainerStarted","Data":"a07c0f7e7fe75792d9d10fea8669a09615eae39da54e1e71ae585e33cf32be9f"} Jul 09 17:10:29.258493 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.258471 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:29.260350 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.260266 2586 generic.go:356] "Generic (PLEG): container finished" podID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" exitCode=0 Jul 09 17:10:29.260509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.260383 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerDied","Data":"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f"} Jul 09 17:10:29.264748 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.264714 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" event={"ID":"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8","Type":"ContainerStarted","Data":"fed33b09a8e5f18784b3b5718daad7fe99efead690523af3d06b07e9e6b5bc4e"} Jul 09 17:10:29.264871 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.264771 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" event={"ID":"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8","Type":"ContainerStarted","Data":"fb0883fba44d51d797e07d3540e277b7abc3a2b410fa8c10424e7282ca131baf"} Jul 09 17:10:29.264871 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.264795 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" event={"ID":"fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8","Type":"ContainerStarted","Data":"5fad9e2f12d18a9df92d1f1a8d8688a970b25a6244202209cb88e31f47702b7d"} Jul 09 17:10:29.266288 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.266267 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" Jul 09 17:10:29.267952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.267803 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" event={"ID":"83b49f60-733f-4618-893a-e2b2f61969f1","Type":"ContainerStarted","Data":"ab80598f27dcc9a81986bb1dee9d226b62917e4c1ecae9e3be1406428d21306d"} Jul 09 17:10:29.267952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.267870 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" event={"ID":"83b49f60-733f-4618-893a-e2b2f61969f1","Type":"ContainerStarted","Data":"e240422c07e09a655480c95f54399382f4ae030be8731e8e8ff1d2b729cdbc6d"} Jul 09 17:10:29.267952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.267912 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" event={"ID":"83b49f60-733f-4618-893a-e2b2f61969f1","Type":"ContainerStarted","Data":"c4006c027b2eefbe216e153a57444ac3b3f16d2e4dc9aa8e1e01ae0c2cb77e58"} Jul 09 17:10:29.270093 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.270069 2586 generic.go:356] "Generic (PLEG): container finished" podID="843c5e17-1dba-405e-a839-a108a442b27f" containerID="b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e" exitCode=0 Jul 09 17:10:29.270183 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.270116 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerDied","Data":"b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e"} Jul 09 17:10:29.277162 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.277115 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" podStartSLOduration=12.381467469 podStartE2EDuration="18.277099137s" totalImagesPullingTime="5.895631668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:29.275579346 +0000 UTC m=+89.205024887" watchObservedRunningTime="2026-07-09 17:10:29.277099137 +0000 UTC m=+89.206544689" Jul 09 17:10:29.351531 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.351480 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f79d4fbc9-pmfsw" podStartSLOduration=11.288869574 podStartE2EDuration="17.351433384s" totalImagesPullingTime="6.06256381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:29.348922295 +0000 UTC m=+89.278367805" watchObservedRunningTime="2026-07-09 17:10:29.351433384 +0000 UTC m=+89.280879087" Jul 09 17:10:29.366864 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:29.366798 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dtzns" podStartSLOduration=11.332750383 podStartE2EDuration="17.366774836s" totalImagesPullingTime="6.034024453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:29.36519006 +0000 UTC m=+89.294635615" watchObservedRunningTime="2026-07-09 17:10:29.366774836 +0000 UTC m=+89.296220350" Jul 09 17:10:32.092772 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.092720 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:32.093133 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.092806 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:32.293614 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.293577 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" event={"ID":"83b49f60-733f-4618-893a-e2b2f61969f1","Type":"ContainerStarted","Data":"40d902afb0d7bddb78caf0055d925bc50e0eebf42a2dc00f31ffdebd6667ac02"} Jul 09 17:10:32.293808 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.293668 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" event={"ID":"83b49f60-733f-4618-893a-e2b2f61969f1","Type":"ContainerStarted","Data":"b96ed871362b8c0abe2003893800c53feadb3e4921d7fcca65acdd85878d42b5"} Jul 09 17:10:32.293808 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.293709 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" event={"ID":"83b49f60-733f-4618-893a-e2b2f61969f1","Type":"ContainerStarted","Data":"ed8505470bcdbeed85ebe0a264ee178b12b6de169a600adbd799994eaa271ba7"} Jul 09 17:10:32.293930 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.293819 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:32.296116 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.296086 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerStarted","Data":"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183"} Jul 09 17:10:32.296245 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.296147 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerStarted","Data":"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c"} Jul 09 17:10:32.316032 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:32.315963 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" podStartSLOduration=13.607239023 podStartE2EDuration="22.31594115s" totalImagesPullingTime="8.708702127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:32.315581093 +0000 UTC m=+92.245026615" watchObservedRunningTime="2026-07-09 17:10:32.31594115 +0000 UTC m=+92.245386662" Jul 09 17:10:33.307723 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:33.307694 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-c8d57cbb5-s7vtg" Jul 09 17:10:34.309399 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:34.309365 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerStarted","Data":"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be"} Jul 09 17:10:34.309862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:34.309459 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerStarted","Data":"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a"} Jul 09 17:10:34.309862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:34.309496 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerStarted","Data":"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d"} Jul 09 17:10:34.309862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:34.309529 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerStarted","Data":"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f"} Jul 09 17:10:34.311961 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:34.311929 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerStarted","Data":"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5"} Jul 09 17:10:34.312082 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:34.312008 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerStarted","Data":"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504"} Jul 09 17:10:34.312082 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:34.312047 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerStarted","Data":"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb"} Jul 09 17:10:34.343976 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:34.343909 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=17.907988366 podStartE2EDuration="26.343886958s" totalImagesPullingTime="8.419398398s" totalInitContainerRuntime="16.500194ms" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:34.341624128 +0000 UTC m=+94.271069667" watchObservedRunningTime="2026-07-09 17:10:34.343886958 +0000 UTC m=+94.273332472" Jul 09 17:10:35.320392 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:35.320351 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerStarted","Data":"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334"} Jul 09 17:10:35.320990 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:35.320424 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerStarted","Data":"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd"} Jul 09 17:10:35.320990 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:35.320467 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerStarted","Data":"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0"} Jul 09 17:10:35.350602 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:35.350539 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=11.61066492 podStartE2EDuration="22.350519086s" totalImagesPullingTime="10.717367551s" totalInitContainerRuntime="22.486615ms" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:35.348321526 +0000 UTC m=+95.277767064" watchObservedRunningTime="2026-07-09 17:10:35.350519086 +0000 UTC m=+95.279964596" Jul 09 17:10:38.950875 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:38.950820 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:10:49.709106 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:49.709033 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f8b57b89-x8vj7" podUID="ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" containerName="console" containerID="cri-o://801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604" gracePeriod=15 Jul 09 17:10:49.998714 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:49.998683 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:50.136652 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.136607 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-service-ca\") pod \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " Jul 09 17:10:50.136652 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.136653 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-oauth-serving-cert\") pod \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " Jul 09 17:10:50.136911 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.136719 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6j7m\" (UniqueName: \"kubernetes.io/projected/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-kube-api-access-t6j7m\") pod \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " Jul 09 17:10:50.136911 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.136752 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-serving-cert\") pod \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " Jul 09 17:10:50.136911 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.136786 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-oauth-config\") pod \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " Jul 09 17:10:50.136911 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.136811 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-config\") pod \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\" (UID: \"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e\") " Jul 09 17:10:50.137110 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.137083 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-service-ca" (OuterVolumeSpecName: "service-ca") pod "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" (UID: "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:10:50.137167 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.137129 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" (UID: "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:10:50.137354 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.137335 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-config" (OuterVolumeSpecName: "console-config") pod "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" (UID: "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:10:50.139210 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.139181 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" (UID: "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:10:50.139417 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.139400 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" (UID: "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:10:50.139508 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.139484 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-kube-api-access-t6j7m" (OuterVolumeSpecName: "kube-api-access-t6j7m") pod "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" (UID: "ecc1e7ce-792f-49c5-8c06-3058f6d25f0e"). InnerVolumeSpecName "kube-api-access-t6j7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:10:50.237988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.237904 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6j7m\" (UniqueName: \"kubernetes.io/projected/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-kube-api-access-t6j7m\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:10:50.237988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.237932 2586 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-serving-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:10:50.237988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.237942 2586 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-oauth-config\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:10:50.237988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.237950 2586 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-console-config\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:10:50.237988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.237959 2586 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:10:50.237988 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.237966 2586 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e-oauth-serving-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:10:50.385581 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.385543 2586 generic.go:356] "Generic (PLEG): container finished" podID="ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" containerID="801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604" exitCode=0 Jul 09 17:10:50.385722 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.385596 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8b57b89-x8vj7" Jul 09 17:10:50.385722 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.385635 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8b57b89-x8vj7" event={"ID":"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e","Type":"ContainerDied","Data":"801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604"} Jul 09 17:10:50.385722 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.385688 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8b57b89-x8vj7" event={"ID":"ecc1e7ce-792f-49c5-8c06-3058f6d25f0e","Type":"ContainerDied","Data":"a3b53f3966b078a67bbb0b9af73bbe699d376369a79840fcd8b6407c1515a044"} Jul 09 17:10:50.385722 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.385709 2586 scope.go:117] "RemoveContainer" containerID="801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604" Jul 09 17:10:50.396947 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.396925 2586 scope.go:117] "RemoveContainer" containerID="801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604" Jul 09 17:10:50.397296 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:10:50.397253 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604\": container with ID starting with 801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604 not found: ID does not exist" containerID="801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604" Jul 09 17:10:50.397358 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.397307 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604"} err="failed to get container status \"801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604\": rpc error: code = NotFound desc = could not find container \"801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604\": container with ID starting with 801ae8748b5e68888416c88af57cfc4d50d9ebe7bb7d1f7101fef165e9d2d604 not found: ID does not exist" Jul 09 17:10:50.413426 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.413396 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f8b57b89-x8vj7"] Jul 09 17:10:50.417327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.417302 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f8b57b89-x8vj7"] Jul 09 17:10:50.671044 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:50.670972 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" path="/var/lib/kubelet/pods/ecc1e7ce-792f-49c5-8c06-3058f6d25f0e/volumes" Jul 09 17:10:52.097860 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:52.097828 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:52.101702 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:52.101680 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-77597554fc-57cgb" Jul 09 17:10:54.226253 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:10:54.226226 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4nvpb" Jul 09 17:11:13.951135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:13.951091 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:14.032751 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:14.032717 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:14.500910 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:14.500880 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:27.776571 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:27.776533 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:11:27.777130 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:27.776883 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="alertmanager" containerID="cri-o://e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c" gracePeriod=120 Jul 09 17:11:27.777130 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:27.776886 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="prom-label-proxy" containerID="cri-o://8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be" gracePeriod=120 Jul 09 17:11:27.777130 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:27.776916 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy-metric" containerID="cri-o://2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a" gracePeriod=120 Jul 09 17:11:27.777130 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:27.776964 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy" containerID="cri-o://b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d" gracePeriod=120 Jul 09 17:11:27.777130 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:27.776944 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="config-reloader" containerID="cri-o://ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183" gracePeriod=120 Jul 09 17:11:27.777130 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:27.776930 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy-web" containerID="cri-o://fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f" gracePeriod=120 Jul 09 17:11:28.542824 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:28.542791 2586 generic.go:356] "Generic (PLEG): container finished" podID="843c5e17-1dba-405e-a839-a108a442b27f" containerID="8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be" exitCode=0 Jul 09 17:11:28.542824 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:28.542815 2586 generic.go:356] "Generic (PLEG): container finished" podID="843c5e17-1dba-405e-a839-a108a442b27f" containerID="b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d" exitCode=0 Jul 09 17:11:28.542824 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:28.542821 2586 generic.go:356] "Generic (PLEG): container finished" podID="843c5e17-1dba-405e-a839-a108a442b27f" containerID="ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183" exitCode=0 Jul 09 17:11:28.542824 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:28.542829 2586 generic.go:356] "Generic (PLEG): container finished" podID="843c5e17-1dba-405e-a839-a108a442b27f" containerID="e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c" exitCode=0 Jul 09 17:11:28.543085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:28.542868 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerDied","Data":"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be"} Jul 09 17:11:28.543085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:28.542933 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerDied","Data":"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d"} Jul 09 17:11:28.543085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:28.542962 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerDied","Data":"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183"} Jul 09 17:11:28.543085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:28.542989 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerDied","Data":"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c"} Jul 09 17:11:29.025589 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.025563 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.087352 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087318 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-cluster-tls-config\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087545 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087362 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087545 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087417 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-main-db\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087545 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087470 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-config-volume\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087545 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087500 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-metrics-client-ca\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087545 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087524 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-tls-assets\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087555 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-web\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087582 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-web-config\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087625 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087659 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087691 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-config-out\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087714 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087748 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh68j\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-kube-api-access-qh68j\") pod \"843c5e17-1dba-405e-a839-a108a442b27f\" (UID: \"843c5e17-1dba-405e-a839-a108a442b27f\") " Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087779 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:11:29.087887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.087852 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:29.088325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.088065 2586 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-main-db\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.088325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.088083 2586 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-metrics-client-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.093275 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.089083 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:29.093894 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.093864 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:29.094010 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.093929 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:11:29.094201 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.094176 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:29.094608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.094579 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-kube-api-access-qh68j" (OuterVolumeSpecName: "kube-api-access-qh68j") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "kube-api-access-qh68j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:11:29.094738 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.094603 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:29.094845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.094798 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-config-out" (OuterVolumeSpecName: "config-out") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:11:29.094845 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.094832 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:29.094933 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.094890 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-config-volume" (OuterVolumeSpecName: "config-volume") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:29.099258 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.099233 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:29.105029 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.105003 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-web-config" (OuterVolumeSpecName: "web-config") pod "843c5e17-1dba-405e-a839-a108a442b27f" (UID: "843c5e17-1dba-405e-a839-a108a442b27f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:29.188675 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188621 2586 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-tls-assets\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188675 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188667 2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188675 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188681 2586 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-web-config\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188675 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188694 2586 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/843c5e17-1dba-405e-a839-a108a442b27f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188706 2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188719 2586 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/843c5e17-1dba-405e-a839-a108a442b27f-config-out\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188731 2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-main-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188742 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh68j\" (UniqueName: \"kubernetes.io/projected/843c5e17-1dba-405e-a839-a108a442b27f-kube-api-access-qh68j\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188754 2586 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-cluster-tls-config\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188765 2586 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.188954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.188778 2586 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/843c5e17-1dba-405e-a839-a108a442b27f-config-volume\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:29.550396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.550356 2586 generic.go:356] "Generic (PLEG): container finished" podID="843c5e17-1dba-405e-a839-a108a442b27f" containerID="2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a" exitCode=0 Jul 09 17:11:29.550396 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.550385 2586 generic.go:356] "Generic (PLEG): container finished" podID="843c5e17-1dba-405e-a839-a108a442b27f" containerID="fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f" exitCode=0 Jul 09 17:11:29.550643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.550453 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerDied","Data":"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a"} Jul 09 17:11:29.550643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.550463 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.550643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.550554 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerDied","Data":"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f"} Jul 09 17:11:29.550643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.550582 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"843c5e17-1dba-405e-a839-a108a442b27f","Type":"ContainerDied","Data":"ae08c6f910f370c8f9d48ee80935ed802b724bae8af8e20ee0d3e4cef3b71840"} Jul 09 17:11:29.550643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.550612 2586 scope.go:117] "RemoveContainer" containerID="8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be" Jul 09 17:11:29.560759 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.560717 2586 scope.go:117] "RemoveContainer" containerID="2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a" Jul 09 17:11:29.568851 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.568830 2586 scope.go:117] "RemoveContainer" containerID="b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d" Jul 09 17:11:29.576000 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.575979 2586 scope.go:117] "RemoveContainer" containerID="fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f" Jul 09 17:11:29.578098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.578046 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:11:29.580804 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.580782 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:11:29.584573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.584554 2586 scope.go:117] "RemoveContainer" containerID="ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183" Jul 09 17:11:29.591516 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.591497 2586 scope.go:117] "RemoveContainer" containerID="e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c" Jul 09 17:11:29.599324 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.599300 2586 scope.go:117] "RemoveContainer" containerID="b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e" Jul 09 17:11:29.605760 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.605740 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:11:29.606092 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606076 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="init-config-reloader" Jul 09 17:11:29.606162 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606096 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="init-config-reloader" Jul 09 17:11:29.606162 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606113 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="config-reloader" Jul 09 17:11:29.606162 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606122 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="config-reloader" Jul 09 17:11:29.606162 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606132 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy-metric" Jul 09 17:11:29.606162 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606140 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy-metric" Jul 09 17:11:29.606162 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606156 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" containerName="console" Jul 09 17:11:29.606162 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606164 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" containerName="console" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606183 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="alertmanager" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606191 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="alertmanager" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606200 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy-web" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606208 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy-web" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606220 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606228 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606238 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="prom-label-proxy" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606246 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="prom-label-proxy" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606333 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy-web" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606344 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy-metric" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606354 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecc1e7ce-792f-49c5-8c06-3058f6d25f0e" containerName="console" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606364 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="kube-rbac-proxy" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606373 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="prom-label-proxy" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606384 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="config-reloader" Jul 09 17:11:29.606593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606393 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="843c5e17-1dba-405e-a839-a108a442b27f" containerName="alertmanager" Jul 09 17:11:29.607388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.606961 2586 scope.go:117] "RemoveContainer" containerID="8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be" Jul 09 17:11:29.607388 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:29.607266 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be\": container with ID starting with 8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be not found: ID does not exist" containerID="8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be" Jul 09 17:11:29.607388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.607292 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be"} err="failed to get container status \"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be\": rpc error: code = NotFound desc = could not find container \"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be\": container with ID starting with 8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be not found: ID does not exist" Jul 09 17:11:29.607388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.607310 2586 scope.go:117] "RemoveContainer" containerID="2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a" Jul 09 17:11:29.607666 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:29.607572 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a\": container with ID starting with 2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a not found: ID does not exist" containerID="2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a" Jul 09 17:11:29.607666 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.607589 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a"} err="failed to get container status \"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a\": rpc error: code = NotFound desc = could not find container \"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a\": container with ID starting with 2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a not found: ID does not exist" Jul 09 17:11:29.607666 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.607601 2586 scope.go:117] "RemoveContainer" containerID="b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d" Jul 09 17:11:29.607858 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:29.607838 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d\": container with ID starting with b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d not found: ID does not exist" containerID="b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d" Jul 09 17:11:29.607915 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.607861 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d"} err="failed to get container status \"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d\": rpc error: code = NotFound desc = could not find container \"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d\": container with ID starting with b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d not found: ID does not exist" Jul 09 17:11:29.607915 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.607878 2586 scope.go:117] "RemoveContainer" containerID="fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f" Jul 09 17:11:29.608105 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:29.608087 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f\": container with ID starting with fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f not found: ID does not exist" containerID="fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f" Jul 09 17:11:29.608160 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.608109 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f"} err="failed to get container status \"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f\": rpc error: code = NotFound desc = could not find container \"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f\": container with ID starting with fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f not found: ID does not exist" Jul 09 17:11:29.608160 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.608124 2586 scope.go:117] "RemoveContainer" containerID="ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183" Jul 09 17:11:29.608369 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:29.608338 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183\": container with ID starting with ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183 not found: ID does not exist" containerID="ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183" Jul 09 17:11:29.608369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.608360 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183"} err="failed to get container status \"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183\": rpc error: code = NotFound desc = could not find container \"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183\": container with ID starting with ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183 not found: ID does not exist" Jul 09 17:11:29.608369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.608373 2586 scope.go:117] "RemoveContainer" containerID="e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c" Jul 09 17:11:29.608660 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:29.608640 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c\": container with ID starting with e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c not found: ID does not exist" containerID="e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c" Jul 09 17:11:29.608769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.608676 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c"} err="failed to get container status \"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c\": rpc error: code = NotFound desc = could not find container \"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c\": container with ID starting with e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c not found: ID does not exist" Jul 09 17:11:29.608769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.608695 2586 scope.go:117] "RemoveContainer" containerID="b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e" Jul 09 17:11:29.608973 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:29.608952 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e\": container with ID starting with b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e not found: ID does not exist" containerID="b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e" Jul 09 17:11:29.609020 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.608980 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e"} err="failed to get container status \"b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e\": rpc error: code = NotFound desc = could not find container \"b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e\": container with ID starting with b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e not found: ID does not exist" Jul 09 17:11:29.609020 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.608999 2586 scope.go:117] "RemoveContainer" containerID="8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be" Jul 09 17:11:29.609240 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.609223 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be"} err="failed to get container status \"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be\": rpc error: code = NotFound desc = could not find container \"8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be\": container with ID starting with 8c0d6983e0db2c53b6900eed1d29bc3dd0478058fa1fc22d1316ca099859d3be not found: ID does not exist" Jul 09 17:11:29.609282 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.609242 2586 scope.go:117] "RemoveContainer" containerID="2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a" Jul 09 17:11:29.609516 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.609487 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a"} err="failed to get container status \"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a\": rpc error: code = NotFound desc = could not find container \"2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a\": container with ID starting with 2fb339d5db503a9f8d907202cc382c8f1dfbf45ce797895337b27bec0164628a not found: ID does not exist" Jul 09 17:11:29.609516 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.609513 2586 scope.go:117] "RemoveContainer" containerID="b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d" Jul 09 17:11:29.609776 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.609757 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d"} err="failed to get container status \"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d\": rpc error: code = NotFound desc = could not find container \"b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d\": container with ID starting with b3ab1c646b47fce8344b132ad33b5f85d2ee587a6d5966f66c608a9d8e6f465d not found: ID does not exist" Jul 09 17:11:29.609843 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.609777 2586 scope.go:117] "RemoveContainer" containerID="fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f" Jul 09 17:11:29.610012 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.609990 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f"} err="failed to get container status \"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f\": rpc error: code = NotFound desc = could not find container \"fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f\": container with ID starting with fecd04ab4efc55b16380063160f91449cdd9a878060c2014f1ab9e8500d6a72f not found: ID does not exist" Jul 09 17:11:29.610050 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.610015 2586 scope.go:117] "RemoveContainer" containerID="ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183" Jul 09 17:11:29.610228 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.610208 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183"} err="failed to get container status \"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183\": rpc error: code = NotFound desc = could not find container \"ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183\": container with ID starting with ad13d34940a7b4ed83d44b58dd385e88eebb403818a4403faf74325582440183 not found: ID does not exist" Jul 09 17:11:29.610290 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.610230 2586 scope.go:117] "RemoveContainer" containerID="e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c" Jul 09 17:11:29.610459 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.610428 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c"} err="failed to get container status \"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c\": rpc error: code = NotFound desc = could not find container \"e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c\": container with ID starting with e30e503fa341984ef89a70f4c8e143a5d1c144b2fe09040aebe74a3b0653115c not found: ID does not exist" Jul 09 17:11:29.610505 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.610461 2586 scope.go:117] "RemoveContainer" containerID="b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e" Jul 09 17:11:29.610641 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.610627 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.610682 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.610669 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e"} err="failed to get container status \"b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e\": rpc error: code = NotFound desc = could not find container \"b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e\": container with ID starting with b4f1a05f718b8358bffc2de824ea965c6e0194129efad7a899941f287dc2033e not found: ID does not exist" Jul 09 17:11:29.615608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615372 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 17:11:29.615608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615372 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 17:11:29.615608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615372 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 17:11:29.615608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615424 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 17:11:29.615608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615431 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 17:11:29.615608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615511 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7sd2g\"" Jul 09 17:11:29.615608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615459 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 17:11:29.616069 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615679 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 17:11:29.616069 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.615787 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 17:11:29.619415 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.619397 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 17:11:29.624130 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.624106 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:11:29.693068 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693033 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bd62717-556d-42cf-9f78-7269904a5684-config-out\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693068 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693066 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693312 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693089 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693312 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693158 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693312 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693211 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bd62717-556d-42cf-9f78-7269904a5684-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693312 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693238 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693458 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693313 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd62717-556d-42cf-9f78-7269904a5684-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693458 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693360 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd62717-556d-42cf-9f78-7269904a5684-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693458 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693382 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twm94\" (UniqueName: \"kubernetes.io/projected/4bd62717-556d-42cf-9f78-7269904a5684-kube-api-access-twm94\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693458 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693409 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4bd62717-556d-42cf-9f78-7269904a5684-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693458 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693427 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-web-config\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693614 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693480 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.693614 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.693503 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-config-volume\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794204 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794167 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd62717-556d-42cf-9f78-7269904a5684-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794397 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794212 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twm94\" (UniqueName: \"kubernetes.io/projected/4bd62717-556d-42cf-9f78-7269904a5684-kube-api-access-twm94\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794397 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794334 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4bd62717-556d-42cf-9f78-7269904a5684-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794397 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794374 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-web-config\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794397 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794395 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794648 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794424 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-config-volume\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794648 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794557 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bd62717-556d-42cf-9f78-7269904a5684-config-out\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794648 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794584 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794648 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794619 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794648 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794645 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794959 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794677 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bd62717-556d-42cf-9f78-7269904a5684-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794959 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794700 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794959 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794748 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd62717-556d-42cf-9f78-7269904a5684-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.794959 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.794773 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4bd62717-556d-42cf-9f78-7269904a5684-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.795539 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.795515 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4bd62717-556d-42cf-9f78-7269904a5684-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.795808 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.795783 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4bd62717-556d-42cf-9f78-7269904a5684-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.797494 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.797466 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4bd62717-556d-42cf-9f78-7269904a5684-config-out\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.797612 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.797590 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-web-config\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.797716 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.797639 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-config-volume\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.798011 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.797983 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.798102 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.798066 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.798390 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.798362 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.798548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.798393 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.798548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.798423 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4bd62717-556d-42cf-9f78-7269904a5684-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.799896 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.799867 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4bd62717-556d-42cf-9f78-7269904a5684-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.803911 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.803860 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twm94\" (UniqueName: \"kubernetes.io/projected/4bd62717-556d-42cf-9f78-7269904a5684-kube-api-access-twm94\") pod \"alertmanager-main-0\" (UID: \"4bd62717-556d-42cf-9f78-7269904a5684\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:29.921729 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:29.921690 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 17:11:30.067967 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:30.067943 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 17:11:30.069964 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:11:30.069937 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bd62717_556d_42cf_9f78_7269904a5684.slice/crio-7926c3aea7ae0daf1f0a66d8e3e1fc3bcd2215289d93c610a06a0a262a493c86 WatchSource:0}: Error finding container 7926c3aea7ae0daf1f0a66d8e3e1fc3bcd2215289d93c610a06a0a262a493c86: Status 404 returned error can't find the container with id 7926c3aea7ae0daf1f0a66d8e3e1fc3bcd2215289d93c610a06a0a262a493c86 Jul 09 17:11:30.561790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:30.561753 2586 generic.go:356] "Generic (PLEG): container finished" podID="4bd62717-556d-42cf-9f78-7269904a5684" containerID="d56b7db8594051d53f05d48f526c332ac726efdb9cd7a78c637aedeea7986f7b" exitCode=0 Jul 09 17:11:30.561959 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:30.561809 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bd62717-556d-42cf-9f78-7269904a5684","Type":"ContainerDied","Data":"d56b7db8594051d53f05d48f526c332ac726efdb9cd7a78c637aedeea7986f7b"} Jul 09 17:11:30.561959 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:30.561927 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bd62717-556d-42cf-9f78-7269904a5684","Type":"ContainerStarted","Data":"7926c3aea7ae0daf1f0a66d8e3e1fc3bcd2215289d93c610a06a0a262a493c86"} Jul 09 17:11:30.672403 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:30.672370 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="843c5e17-1dba-405e-a839-a108a442b27f" path="/var/lib/kubelet/pods/843c5e17-1dba-405e-a839-a108a442b27f/volumes" Jul 09 17:11:31.569809 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:31.569774 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bd62717-556d-42cf-9f78-7269904a5684","Type":"ContainerStarted","Data":"45cd22f8a21cebbe6f27c2e0c875943af383a349b88fedefb1dab716a0723455"} Jul 09 17:11:31.570239 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:31.569835 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bd62717-556d-42cf-9f78-7269904a5684","Type":"ContainerStarted","Data":"a35e29b41145d472282489aca5e3cda8d137259a18ab1715346ef61bd859c394"} Jul 09 17:11:31.570239 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:31.569868 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bd62717-556d-42cf-9f78-7269904a5684","Type":"ContainerStarted","Data":"920023e3a48842260fadb744e66053113c2b68255da882c7901609d048a9e53d"} Jul 09 17:11:31.570239 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:31.569894 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bd62717-556d-42cf-9f78-7269904a5684","Type":"ContainerStarted","Data":"06e2963f863888325cf6a87ba2f5e29dafbccb2f633965815b8b4b68a924aebb"} Jul 09 17:11:31.570239 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:31.569917 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bd62717-556d-42cf-9f78-7269904a5684","Type":"ContainerStarted","Data":"05888d22329d526caf94910bffc0bac0f87910efd6a9aa26cb3df0b0ff8ce2e7"} Jul 09 17:11:31.570239 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:31.569940 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4bd62717-556d-42cf-9f78-7269904a5684","Type":"ContainerStarted","Data":"1e82bf952e40f37107e54a650d1bedffe2556c50d7d6417d42b7db73751ee656"} Jul 09 17:11:31.600635 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:31.600559 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.582183614 podStartE2EDuration="2.600543339s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.359725ms" isStatefulPod=false podCreationTimestamp="2026-07-09 17:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:11:31.59867307 +0000 UTC m=+151.528118583" watchObservedRunningTime="2026-07-09 17:11:31.600543339 +0000 UTC m=+151.529988850" Jul 09 17:11:32.061273 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.061231 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:11:32.061640 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.061602 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="prometheus" containerID="cri-o://6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" gracePeriod=600 Jul 09 17:11:32.061805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.061626 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy" containerID="cri-o://3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" gracePeriod=600 Jul 09 17:11:32.061805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.061652 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy-thanos" containerID="cri-o://81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" gracePeriod=600 Jul 09 17:11:32.061805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.061673 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="thanos-sidecar" containerID="cri-o://848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" gracePeriod=600 Jul 09 17:11:32.061805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.061692 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy-web" containerID="cri-o://0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" gracePeriod=600 Jul 09 17:11:32.061805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.061764 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="config-reloader" containerID="cri-o://68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" gracePeriod=600 Jul 09 17:11:32.309782 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.309467 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.417859 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.417772 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-tls-assets\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.417859 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.417811 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.417859 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.417837 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-tls\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.417877 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-0\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.417896 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-serving-certs-ca-bundle\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.417919 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-thanos-prometheus-http-client-file\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.417944 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-db\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.417972 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsc87\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-kube-api-access-qsc87\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418014 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-1\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418050 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-trusted-ca-bundle\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418087 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-metrics-client-ca\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418124 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-grpc-tls\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418153 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418180 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-config-out\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418207 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-config\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418231 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-metrics-client-certs\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418267 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-2\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418326 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-kubelet-serving-ca-bundle\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418379 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-kube-rbac-proxy\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.418607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418417 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-web-config\") pod \"7c830588-91f7-4962-bc87-2b5a3c21a06f\" (UID: \"7c830588-91f7-4962-bc87-2b5a3c21a06f\") " Jul 09 17:11:32.420599 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.418315 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:32.420599 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.419099 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:11:32.420797 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.419992 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:32.420797 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.420546 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:32.421490 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.421414 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:32.421799 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.421772 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:32.422032 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.421803 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:32.422136 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.422113 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:32.422590 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.422555 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.423140 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.422726 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.423140 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.422748 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.428110 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.427788 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.428110 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.427834 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-kube-api-access-qsc87" (OuterVolumeSpecName: "kube-api-access-qsc87") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "kube-api-access-qsc87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:11:32.428660 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.428607 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-config" (OuterVolumeSpecName: "config") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.429039 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.429011 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.429039 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.429026 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.429199 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.429050 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:11:32.429199 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.429068 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.429296 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.429197 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-config-out" (OuterVolumeSpecName: "config-out") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:11:32.437848 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.437820 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-web-config" (OuterVolumeSpecName: "web-config") pod "7c830588-91f7-4962-bc87-2b5a3c21a06f" (UID: "7c830588-91f7-4962-bc87-2b5a3c21a06f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:32.519997 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.519948 2586 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-web-config\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.519997 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.519990 2586 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-tls-assets\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.519997 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520002 2586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.519997 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520012 2586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520022 2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520031 2586 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520040 2586 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520048 2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-db\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520057 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsc87\" (UniqueName: \"kubernetes.io/projected/7c830588-91f7-4962-bc87-2b5a3c21a06f-kube-api-access-qsc87\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520065 2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520073 2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520084 2586 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-metrics-client-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520092 2586 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-grpc-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520100 2586 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520108 2586 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7c830588-91f7-4962-bc87-2b5a3c21a06f-config-out\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520116 2586 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-config\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520123 2586 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-metrics-client-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520131 2586 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520139 2586 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c830588-91f7-4962-bc87-2b5a3c21a06f-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.520265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.520147 2586 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7c830588-91f7-4962-bc87-2b5a3c21a06f-secret-kube-rbac-proxy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:11:32.576484 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576420 2586 generic.go:356] "Generic (PLEG): container finished" podID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" exitCode=0 Jul 09 17:11:32.576484 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576474 2586 generic.go:356] "Generic (PLEG): container finished" podID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" exitCode=0 Jul 09 17:11:32.576484 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576486 2586 generic.go:356] "Generic (PLEG): container finished" podID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" exitCode=0 Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576498 2586 generic.go:356] "Generic (PLEG): container finished" podID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" exitCode=0 Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576507 2586 generic.go:356] "Generic (PLEG): container finished" podID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" exitCode=0 Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576505 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerDied","Data":"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334"} Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576546 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576516 2586 generic.go:356] "Generic (PLEG): container finished" podID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" exitCode=0 Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576607 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerDied","Data":"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd"} Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576636 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerDied","Data":"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0"} Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576662 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerDied","Data":"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5"} Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576687 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerDied","Data":"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504"} Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576712 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerDied","Data":"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb"} Jul 09 17:11:32.576895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576736 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7c830588-91f7-4962-bc87-2b5a3c21a06f","Type":"ContainerDied","Data":"2b96cf6defbb3259ae7f65f05ec83abf759571593ca41b9342f24531fa2ffc35"} Jul 09 17:11:32.577599 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.576683 2586 scope.go:117] "RemoveContainer" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" Jul 09 17:11:32.586292 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.586271 2586 scope.go:117] "RemoveContainer" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" Jul 09 17:11:32.593699 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.593681 2586 scope.go:117] "RemoveContainer" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" Jul 09 17:11:32.602923 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.602900 2586 scope.go:117] "RemoveContainer" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" Jul 09 17:11:32.603483 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.603418 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:11:32.605755 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.605732 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:11:32.610904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.610888 2586 scope.go:117] "RemoveContainer" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" Jul 09 17:11:32.618428 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.618409 2586 scope.go:117] "RemoveContainer" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" Jul 09 17:11:32.626577 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.626552 2586 scope.go:117] "RemoveContainer" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" Jul 09 17:11:32.634295 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634238 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:11:32.634360 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634331 2586 scope.go:117] "RemoveContainer" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" Jul 09 17:11:32.634691 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634674 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634698 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634712 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="config-reloader" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634721 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="config-reloader" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634736 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="thanos-sidecar" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634744 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="thanos-sidecar" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634755 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="prometheus" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634762 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="prometheus" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634772 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy-thanos" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:32.634670 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": container with ID starting with 81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334 not found: ID does not exist" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" Jul 09 17:11:32.634783 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634780 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy-thanos" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634792 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy-web" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634800 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy-web" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634812 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="init-config-reloader" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634802 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334"} err="failed to get container status \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": rpc error: code = NotFound desc = could not find container \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": container with ID starting with 81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334 not found: ID does not exist" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634832 2586 scope.go:117] "RemoveContainer" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634820 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="init-config-reloader" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.634990 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635006 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy-thanos" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635018 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="thanos-sidecar" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635028 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="prometheus" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635037 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="kube-rbac-proxy-web" Jul 09 17:11:32.635218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635046 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" containerName="config-reloader" Jul 09 17:11:32.635773 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:32.635193 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": container with ID starting with 3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd not found: ID does not exist" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" Jul 09 17:11:32.635773 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635270 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd"} err="failed to get container status \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": rpc error: code = NotFound desc = could not find container \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": container with ID starting with 3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd not found: ID does not exist" Jul 09 17:11:32.635773 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635306 2586 scope.go:117] "RemoveContainer" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" Jul 09 17:11:32.635773 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:32.635666 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": container with ID starting with 0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0 not found: ID does not exist" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" Jul 09 17:11:32.635773 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635694 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0"} err="failed to get container status \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": rpc error: code = NotFound desc = could not find container \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": container with ID starting with 0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0 not found: ID does not exist" Jul 09 17:11:32.635773 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.635715 2586 scope.go:117] "RemoveContainer" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" Jul 09 17:11:32.636006 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:32.635993 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": container with ID starting with 848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5 not found: ID does not exist" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" Jul 09 17:11:32.636039 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.636013 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5"} err="failed to get container status \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": rpc error: code = NotFound desc = could not find container \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": container with ID starting with 848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5 not found: ID does not exist" Jul 09 17:11:32.636039 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.636029 2586 scope.go:117] "RemoveContainer" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" Jul 09 17:11:32.636278 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:32.636261 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": container with ID starting with 68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504 not found: ID does not exist" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" Jul 09 17:11:32.636342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.636287 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504"} err="failed to get container status \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": rpc error: code = NotFound desc = could not find container \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": container with ID starting with 68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504 not found: ID does not exist" Jul 09 17:11:32.636342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.636308 2586 scope.go:117] "RemoveContainer" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" Jul 09 17:11:32.636588 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:32.636564 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": container with ID starting with 6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb not found: ID does not exist" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" Jul 09 17:11:32.636642 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.636597 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb"} err="failed to get container status \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": rpc error: code = NotFound desc = could not find container \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": container with ID starting with 6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb not found: ID does not exist" Jul 09 17:11:32.636642 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.636618 2586 scope.go:117] "RemoveContainer" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" Jul 09 17:11:32.636880 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:11:32.636862 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": container with ID starting with 9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f not found: ID does not exist" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" Jul 09 17:11:32.636943 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.636888 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f"} err="failed to get container status \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": rpc error: code = NotFound desc = could not find container \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": container with ID starting with 9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f not found: ID does not exist" Jul 09 17:11:32.636943 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.636908 2586 scope.go:117] "RemoveContainer" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" Jul 09 17:11:32.637159 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.637143 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334"} err="failed to get container status \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": rpc error: code = NotFound desc = could not find container \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": container with ID starting with 81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334 not found: ID does not exist" Jul 09 17:11:32.637199 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.637161 2586 scope.go:117] "RemoveContainer" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" Jul 09 17:11:32.637368 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.637350 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd"} err="failed to get container status \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": rpc error: code = NotFound desc = could not find container \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": container with ID starting with 3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd not found: ID does not exist" Jul 09 17:11:32.637429 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.637369 2586 scope.go:117] "RemoveContainer" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" Jul 09 17:11:32.637637 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.637619 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0"} err="failed to get container status \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": rpc error: code = NotFound desc = could not find container \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": container with ID starting with 0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0 not found: ID does not exist" Jul 09 17:11:32.637688 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.637637 2586 scope.go:117] "RemoveContainer" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" Jul 09 17:11:32.637859 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.637843 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5"} err="failed to get container status \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": rpc error: code = NotFound desc = could not find container \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": container with ID starting with 848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5 not found: ID does not exist" Jul 09 17:11:32.637918 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.637861 2586 scope.go:117] "RemoveContainer" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" Jul 09 17:11:32.638085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638069 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504"} err="failed to get container status \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": rpc error: code = NotFound desc = could not find container \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": container with ID starting with 68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504 not found: ID does not exist" Jul 09 17:11:32.638127 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638086 2586 scope.go:117] "RemoveContainer" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" Jul 09 17:11:32.638268 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638252 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb"} err="failed to get container status \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": rpc error: code = NotFound desc = could not find container \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": container with ID starting with 6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb not found: ID does not exist" Jul 09 17:11:32.638305 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638269 2586 scope.go:117] "RemoveContainer" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" Jul 09 17:11:32.638461 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638430 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f"} err="failed to get container status \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": rpc error: code = NotFound desc = could not find container \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": container with ID starting with 9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f not found: ID does not exist" Jul 09 17:11:32.638544 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638464 2586 scope.go:117] "RemoveContainer" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" Jul 09 17:11:32.638707 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638691 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334"} err="failed to get container status \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": rpc error: code = NotFound desc = could not find container \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": container with ID starting with 81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334 not found: ID does not exist" Jul 09 17:11:32.638773 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638708 2586 scope.go:117] "RemoveContainer" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" Jul 09 17:11:32.638935 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638918 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd"} err="failed to get container status \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": rpc error: code = NotFound desc = could not find container \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": container with ID starting with 3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd not found: ID does not exist" Jul 09 17:11:32.638977 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.638936 2586 scope.go:117] "RemoveContainer" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" Jul 09 17:11:32.639131 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.639113 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0"} err="failed to get container status \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": rpc error: code = NotFound desc = could not find container \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": container with ID starting with 0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0 not found: ID does not exist" Jul 09 17:11:32.639200 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.639132 2586 scope.go:117] "RemoveContainer" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" Jul 09 17:11:32.639316 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.639296 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5"} err="failed to get container status \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": rpc error: code = NotFound desc = could not find container \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": container with ID starting with 848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5 not found: ID does not exist" Jul 09 17:11:32.639365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.639319 2586 scope.go:117] "RemoveContainer" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" Jul 09 17:11:32.639530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.639503 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504"} err="failed to get container status \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": rpc error: code = NotFound desc = could not find container \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": container with ID starting with 68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504 not found: ID does not exist" Jul 09 17:11:32.639530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.639529 2586 scope.go:117] "RemoveContainer" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" Jul 09 17:11:32.639777 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.639755 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb"} err="failed to get container status \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": rpc error: code = NotFound desc = could not find container \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": container with ID starting with 6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb not found: ID does not exist" Jul 09 17:11:32.639986 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.639779 2586 scope.go:117] "RemoveContainer" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" Jul 09 17:11:32.640060 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.640036 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f"} err="failed to get container status \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": rpc error: code = NotFound desc = could not find container \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": container with ID starting with 9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f not found: ID does not exist" Jul 09 17:11:32.640114 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.640061 2586 scope.go:117] "RemoveContainer" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" Jul 09 17:11:32.640114 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.640068 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.640404 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.640376 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334"} err="failed to get container status \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": rpc error: code = NotFound desc = could not find container \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": container with ID starting with 81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334 not found: ID does not exist" Jul 09 17:11:32.640546 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.640405 2586 scope.go:117] "RemoveContainer" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" Jul 09 17:11:32.640817 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.640793 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd"} err="failed to get container status \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": rpc error: code = NotFound desc = could not find container \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": container with ID starting with 3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd not found: ID does not exist" Jul 09 17:11:32.640881 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.640820 2586 scope.go:117] "RemoveContainer" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" Jul 09 17:11:32.641099 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.641079 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0"} err="failed to get container status \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": rpc error: code = NotFound desc = could not find container \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": container with ID starting with 0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0 not found: ID does not exist" Jul 09 17:11:32.641150 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.641101 2586 scope.go:117] "RemoveContainer" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" Jul 09 17:11:32.641326 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.641311 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5"} err="failed to get container status \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": rpc error: code = NotFound desc = could not find container \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": container with ID starting with 848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5 not found: ID does not exist" Jul 09 17:11:32.641374 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.641327 2586 scope.go:117] "RemoveContainer" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" Jul 09 17:11:32.641569 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.641542 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504"} err="failed to get container status \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": rpc error: code = NotFound desc = could not find container \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": container with ID starting with 68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504 not found: ID does not exist" Jul 09 17:11:32.641644 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.641572 2586 scope.go:117] "RemoveContainer" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" Jul 09 17:11:32.641802 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.641773 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb"} err="failed to get container status \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": rpc error: code = NotFound desc = could not find container \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": container with ID starting with 6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb not found: ID does not exist" Jul 09 17:11:32.641802 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.641799 2586 scope.go:117] "RemoveContainer" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" Jul 09 17:11:32.642071 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642036 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f"} err="failed to get container status \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": rpc error: code = NotFound desc = could not find container \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": container with ID starting with 9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f not found: ID does not exist" Jul 09 17:11:32.642071 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642057 2586 scope.go:117] "RemoveContainer" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" Jul 09 17:11:32.642325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642259 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334"} err="failed to get container status \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": rpc error: code = NotFound desc = could not find container \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": container with ID starting with 81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334 not found: ID does not exist" Jul 09 17:11:32.642325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642275 2586 scope.go:117] "RemoveContainer" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" Jul 09 17:11:32.642542 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642514 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd"} err="failed to get container status \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": rpc error: code = NotFound desc = could not find container \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": container with ID starting with 3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd not found: ID does not exist" Jul 09 17:11:32.642542 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642541 2586 scope.go:117] "RemoveContainer" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" Jul 09 17:11:32.642792 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642775 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0"} err="failed to get container status \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": rpc error: code = NotFound desc = could not find container \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": container with ID starting with 0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0 not found: ID does not exist" Jul 09 17:11:32.642877 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642792 2586 scope.go:117] "RemoveContainer" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" Jul 09 17:11:32.643034 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.642987 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5"} err="failed to get container status \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": rpc error: code = NotFound desc = could not find container \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": container with ID starting with 848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5 not found: ID does not exist" Jul 09 17:11:32.643034 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643013 2586 scope.go:117] "RemoveContainer" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" Jul 09 17:11:32.643353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643269 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 17:11:32.643353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643295 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 17:11:32.643353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643296 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7cli00m0kjqqp\"" Jul 09 17:11:32.643353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643265 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504"} err="failed to get container status \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": rpc error: code = NotFound desc = could not find container \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": container with ID starting with 68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504 not found: ID does not exist" Jul 09 17:11:32.643353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643349 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 17:11:32.643353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643302 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 17:11:32.643661 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643388 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 17:11:32.643661 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643347 2586 scope.go:117] "RemoveContainer" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" Jul 09 17:11:32.643661 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643493 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 17:11:32.643768 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643689 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb"} err="failed to get container status \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": rpc error: code = NotFound desc = could not find container \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": container with ID starting with 6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb not found: ID does not exist" Jul 09 17:11:32.643768 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643711 2586 scope.go:117] "RemoveContainer" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" Jul 09 17:11:32.643859 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643773 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 17:11:32.643859 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.643775 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 17:11:32.644193 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644169 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f"} err="failed to get container status \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": rpc error: code = NotFound desc = could not find container \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": container with ID starting with 9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f not found: ID does not exist" Jul 09 17:11:32.644262 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644194 2586 scope.go:117] "RemoveContainer" containerID="81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334" Jul 09 17:11:32.644426 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644408 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334"} err="failed to get container status \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": rpc error: code = NotFound desc = could not find container \"81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334\": container with ID starting with 81eef8a03bdaedd2dc3ca504d43438697dcb51f2ef9677d83666fb26b8f45334 not found: ID does not exist" Jul 09 17:11:32.644530 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644428 2586 scope.go:117] "RemoveContainer" containerID="3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd" Jul 09 17:11:32.644716 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644693 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd"} err="failed to get container status \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": rpc error: code = NotFound desc = could not find container \"3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd\": container with ID starting with 3ba92c00b564a117b72ffdbf6fa9685b03a1516b0f4e161151b309b8540385dd not found: ID does not exist" Jul 09 17:11:32.644791 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644725 2586 scope.go:117] "RemoveContainer" containerID="0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0" Jul 09 17:11:32.644852 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644791 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 17:11:32.644903 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644855 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 17:11:32.644969 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.644952 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-94drg\"" Jul 09 17:11:32.645022 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.645010 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 17:11:32.645076 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.645047 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 17:11:32.645454 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.645411 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0"} err="failed to get container status \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": rpc error: code = NotFound desc = could not find container \"0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0\": container with ID starting with 0d909ff001d117dfe9d337fcef7f1fc461c45fb8814843de0fbb0661ff28dec0 not found: ID does not exist" Jul 09 17:11:32.645543 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.645455 2586 scope.go:117] "RemoveContainer" containerID="848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5" Jul 09 17:11:32.645849 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.645825 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5"} err="failed to get container status \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": rpc error: code = NotFound desc = could not find container \"848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5\": container with ID starting with 848f065a31727d0bd7eea716f53a24acb85b7ba0534e9802a5e51ee0e3c596d5 not found: ID does not exist" Jul 09 17:11:32.645922 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.645850 2586 scope.go:117] "RemoveContainer" containerID="68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504" Jul 09 17:11:32.646125 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.646104 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504"} err="failed to get container status \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": rpc error: code = NotFound desc = could not find container \"68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504\": container with ID starting with 68061efd6e8a71719b538d2756bb19434b5295bcc773e8831ff786d3956a8504 not found: ID does not exist" Jul 09 17:11:32.646204 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.646126 2586 scope.go:117] "RemoveContainer" containerID="6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb" Jul 09 17:11:32.646494 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.646427 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb"} err="failed to get container status \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": rpc error: code = NotFound desc = could not find container \"6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb\": container with ID starting with 6d202395d179dec04f04a1fb67c16c0f70f9c5ea662db8ce06704ea837f861eb not found: ID does not exist" Jul 09 17:11:32.646494 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.646473 2586 scope.go:117] "RemoveContainer" containerID="9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f" Jul 09 17:11:32.646772 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.646749 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f"} err="failed to get container status \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": rpc error: code = NotFound desc = could not find container \"9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f\": container with ID starting with 9ae262dac39e6dfb3c7b0d92d046da9e008ac8c6f954a43cc823f90764465e9f not found: ID does not exist" Jul 09 17:11:32.647377 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.647357 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 17:11:32.649600 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.649581 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 17:11:32.658004 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.657979 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:11:32.672328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.672259 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c830588-91f7-4962-bc87-2b5a3c21a06f" path="/var/lib/kubelet/pods/7c830588-91f7-4962-bc87-2b5a3c21a06f/volumes" Jul 09 17:11:32.721829 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.721785 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.721829 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.721822 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722048 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.721939 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722048 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.721970 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722048 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.721998 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv8d7\" (UniqueName: \"kubernetes.io/projected/1f109601-c3bb-431e-9dfa-97a15be272c7-kube-api-access-qv8d7\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722048 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722046 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722183 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722081 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722183 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722112 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-config\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722183 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722127 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722286 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722206 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f109601-c3bb-431e-9dfa-97a15be272c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722279 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722310 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722489 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722469 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722570 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722512 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722569 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722701 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722684 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722738 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722713 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f109601-c3bb-431e-9dfa-97a15be272c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722738 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722732 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722800 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722765 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.722831 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.722823 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.823822 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.823784 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824009 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.823839 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-config\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824009 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.823859 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824009 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.823915 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f109601-c3bb-431e-9dfa-97a15be272c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824009 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.823958 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824009 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.823981 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824252 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824016 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824252 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824052 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824252 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824081 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824398 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824273 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824398 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824315 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f109601-c3bb-431e-9dfa-97a15be272c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824398 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824340 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824398 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824369 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824421 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824472 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824500 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824559 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824584 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824600 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824615 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8d7\" (UniqueName: \"kubernetes.io/projected/1f109601-c3bb-431e-9dfa-97a15be272c7-kube-api-access-qv8d7\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824939 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824644 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.824939 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824662 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.825038 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.824936 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.825674 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.825358 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.825791 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.825749 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.827274 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.826915 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-config\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.828326 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.827422 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.828326 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.827825 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.828326 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.827923 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.828326 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.828285 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1f109601-c3bb-431e-9dfa-97a15be272c7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.828620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.828597 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.828675 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.828646 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1f109601-c3bb-431e-9dfa-97a15be272c7-config-out\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.828738 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.828713 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.828991 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.828962 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1f109601-c3bb-431e-9dfa-97a15be272c7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.829092 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.829054 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.829555 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.829535 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.830827 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.830802 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.830902 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.830859 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-web-config\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.831149 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.831133 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1f109601-c3bb-431e-9dfa-97a15be272c7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.834449 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.834411 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv8d7\" (UniqueName: \"kubernetes.io/projected/1f109601-c3bb-431e-9dfa-97a15be272c7-kube-api-access-qv8d7\") pod \"prometheus-k8s-0\" (UID: \"1f109601-c3bb-431e-9dfa-97a15be272c7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:32.951826 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:32.951737 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:11:33.100365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:33.100221 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 17:11:33.102906 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:11:33.102874 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f109601_c3bb_431e_9dfa_97a15be272c7.slice/crio-b455eefac39cc86d0056a8d6e44800dfa287843772abc78ec8cbcacb436b8a1e WatchSource:0}: Error finding container b455eefac39cc86d0056a8d6e44800dfa287843772abc78ec8cbcacb436b8a1e: Status 404 returned error can't find the container with id b455eefac39cc86d0056a8d6e44800dfa287843772abc78ec8cbcacb436b8a1e Jul 09 17:11:33.582732 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:33.582690 2586 generic.go:356] "Generic (PLEG): container finished" podID="1f109601-c3bb-431e-9dfa-97a15be272c7" containerID="0794e28e1121451cad1bcef683ddcdada6de3deda1cc189bc60b8a6bfff5d982" exitCode=0 Jul 09 17:11:33.582732 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:33.582730 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f109601-c3bb-431e-9dfa-97a15be272c7","Type":"ContainerDied","Data":"0794e28e1121451cad1bcef683ddcdada6de3deda1cc189bc60b8a6bfff5d982"} Jul 09 17:11:33.583193 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:33.582840 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f109601-c3bb-431e-9dfa-97a15be272c7","Type":"ContainerStarted","Data":"b455eefac39cc86d0056a8d6e44800dfa287843772abc78ec8cbcacb436b8a1e"} Jul 09 17:11:34.592302 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:34.592267 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f109601-c3bb-431e-9dfa-97a15be272c7","Type":"ContainerStarted","Data":"2c16aa87a2ea2f33016240e3b0f39f9ef207fa466d830a4d7acba9f47d8ce226"} Jul 09 17:11:34.592718 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:34.592333 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f109601-c3bb-431e-9dfa-97a15be272c7","Type":"ContainerStarted","Data":"b723e4e8e12666a715a770c5fca3936cead7ecafbd712fd778f3eaf9628e4e68"} Jul 09 17:11:34.592718 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:34.592361 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f109601-c3bb-431e-9dfa-97a15be272c7","Type":"ContainerStarted","Data":"ec924c28136799f8ec5de5e987a224e38d938c0c0c031189ec65f450e8aec4a2"} Jul 09 17:11:34.592718 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:34.592388 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f109601-c3bb-431e-9dfa-97a15be272c7","Type":"ContainerStarted","Data":"9e105c6420591f97b87615e0721ba05b8b1ab3269ca133f5dfbd54bce68cec16"} Jul 09 17:11:34.592718 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:34.592412 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f109601-c3bb-431e-9dfa-97a15be272c7","Type":"ContainerStarted","Data":"24eead8f73748a30532327a7f9d1a3af0b8a0633a087938c272bb7f51105be1c"} Jul 09 17:11:34.592718 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:34.592434 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1f109601-c3bb-431e-9dfa-97a15be272c7","Type":"ContainerStarted","Data":"c870b257935529a3864acf251eba894181673e99df92180272551f7b44522531"} Jul 09 17:11:34.621864 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:34.621799 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.602717487 podStartE2EDuration="2.621778102s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.060615ms" isStatefulPod=false podCreationTimestamp="2026-07-09 17:11:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:11:34.6194198 +0000 UTC m=+154.548865333" watchObservedRunningTime="2026-07-09 17:11:34.621778102 +0000 UTC m=+154.551223615" Jul 09 17:11:37.952121 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:11:37.952080 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:12:32.952618 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:12:32.952585 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:12:32.968075 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:12:32.968046 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:12:33.854937 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:12:33.854912 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 17:13:24.983473 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:24.983422 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7crvk"] Jul 09 17:13:24.985784 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:24.985768 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" Jul 09 17:13:24.988851 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:24.988827 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 17:13:24.988980 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:24.988930 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mv966\"" Jul 09 17:13:24.990294 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:24.990276 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 17:13:24.995784 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:24.995761 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7crvk"] Jul 09 17:13:25.030768 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:25.030725 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf6nt\" (UniqueName: \"kubernetes.io/projected/d3d551c1-bfc9-4117-a3f4-635a5e177622-kube-api-access-bf6nt\") pod \"cert-manager-cainjector-559cdd8c87-7crvk\" (UID: \"d3d551c1-bfc9-4117-a3f4-635a5e177622\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" Jul 09 17:13:25.030943 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:25.030789 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3d551c1-bfc9-4117-a3f4-635a5e177622-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7crvk\" (UID: \"d3d551c1-bfc9-4117-a3f4-635a5e177622\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" Jul 09 17:13:25.132229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:25.132188 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3d551c1-bfc9-4117-a3f4-635a5e177622-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7crvk\" (UID: \"d3d551c1-bfc9-4117-a3f4-635a5e177622\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" Jul 09 17:13:25.132426 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:25.132292 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf6nt\" (UniqueName: \"kubernetes.io/projected/d3d551c1-bfc9-4117-a3f4-635a5e177622-kube-api-access-bf6nt\") pod \"cert-manager-cainjector-559cdd8c87-7crvk\" (UID: \"d3d551c1-bfc9-4117-a3f4-635a5e177622\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" Jul 09 17:13:25.141933 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:25.141904 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3d551c1-bfc9-4117-a3f4-635a5e177622-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7crvk\" (UID: \"d3d551c1-bfc9-4117-a3f4-635a5e177622\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" Jul 09 17:13:25.142078 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:25.142059 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf6nt\" (UniqueName: \"kubernetes.io/projected/d3d551c1-bfc9-4117-a3f4-635a5e177622-kube-api-access-bf6nt\") pod \"cert-manager-cainjector-559cdd8c87-7crvk\" (UID: \"d3d551c1-bfc9-4117-a3f4-635a5e177622\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" Jul 09 17:13:25.307432 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:25.307345 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" Jul 09 17:13:25.435082 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:25.435058 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7crvk"] Jul 09 17:13:25.437150 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:13:25.437118 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3d551c1_bfc9_4117_a3f4_635a5e177622.slice/crio-eadc2dd1c8f64695a1901a13306bc2bc25b5fe22251ec8efce6927b1dfae66a4 WatchSource:0}: Error finding container eadc2dd1c8f64695a1901a13306bc2bc25b5fe22251ec8efce6927b1dfae66a4: Status 404 returned error can't find the container with id eadc2dd1c8f64695a1901a13306bc2bc25b5fe22251ec8efce6927b1dfae66a4 Jul 09 17:13:26.045415 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:26.045374 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" event={"ID":"d3d551c1-bfc9-4117-a3f4-635a5e177622","Type":"ContainerStarted","Data":"eadc2dd1c8f64695a1901a13306bc2bc25b5fe22251ec8efce6927b1dfae66a4"} Jul 09 17:13:29.060080 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.060044 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" event={"ID":"d3d551c1-bfc9-4117-a3f4-635a5e177622","Type":"ContainerStarted","Data":"f6ce3dfdd1b0c1684e7a25a633d81cca33bf1a4f57a765fe850594fdf2430860"} Jul 09 17:13:29.077098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.077042 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7crvk" podStartSLOduration=2.090027454 podStartE2EDuration="5.077024503s" totalImagesPullingTime="2.986997049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:13:29.075810384 +0000 UTC m=+269.005255919" watchObservedRunningTime="2026-07-09 17:13:29.077024503 +0000 UTC m=+269.006470014" Jul 09 17:13:29.615458 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.615406 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr"] Jul 09 17:13:29.617743 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.617728 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" Jul 09 17:13:29.620710 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.620688 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 17:13:29.620836 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.620729 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:13:29.620836 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.620803 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rgq69\"" Jul 09 17:13:29.628319 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.628297 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr"] Jul 09 17:13:29.673123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.673084 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj2gb\" (UniqueName: \"kubernetes.io/projected/b7cc259f-0af4-4177-b4fc-328d68e08936-kube-api-access-bj2gb\") pod \"openshift-lws-operator-fd8ccff4c-jgxgr\" (UID: \"b7cc259f-0af4-4177-b4fc-328d68e08936\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" Jul 09 17:13:29.673276 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.673132 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7cc259f-0af4-4177-b4fc-328d68e08936-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jgxgr\" (UID: \"b7cc259f-0af4-4177-b4fc-328d68e08936\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" Jul 09 17:13:29.774114 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.774084 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj2gb\" (UniqueName: \"kubernetes.io/projected/b7cc259f-0af4-4177-b4fc-328d68e08936-kube-api-access-bj2gb\") pod \"openshift-lws-operator-fd8ccff4c-jgxgr\" (UID: \"b7cc259f-0af4-4177-b4fc-328d68e08936\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" Jul 09 17:13:29.774257 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.774134 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7cc259f-0af4-4177-b4fc-328d68e08936-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jgxgr\" (UID: \"b7cc259f-0af4-4177-b4fc-328d68e08936\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" Jul 09 17:13:29.774480 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.774465 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7cc259f-0af4-4177-b4fc-328d68e08936-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jgxgr\" (UID: \"b7cc259f-0af4-4177-b4fc-328d68e08936\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" Jul 09 17:13:29.784868 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.784845 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj2gb\" (UniqueName: \"kubernetes.io/projected/b7cc259f-0af4-4177-b4fc-328d68e08936-kube-api-access-bj2gb\") pod \"openshift-lws-operator-fd8ccff4c-jgxgr\" (UID: \"b7cc259f-0af4-4177-b4fc-328d68e08936\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" Jul 09 17:13:29.926517 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:29.926415 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" Jul 09 17:13:30.051481 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:30.051431 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr"] Jul 09 17:13:30.054243 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:13:30.054218 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7cc259f_0af4_4177_b4fc_328d68e08936.slice/crio-138d0dd447390fb20466b8bf382ba3b163e589408f672249416fce2ed6ad17ce WatchSource:0}: Error finding container 138d0dd447390fb20466b8bf382ba3b163e589408f672249416fce2ed6ad17ce: Status 404 returned error can't find the container with id 138d0dd447390fb20466b8bf382ba3b163e589408f672249416fce2ed6ad17ce Jul 09 17:13:30.064870 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:30.064840 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" event={"ID":"b7cc259f-0af4-4177-b4fc-328d68e08936","Type":"ContainerStarted","Data":"138d0dd447390fb20466b8bf382ba3b163e589408f672249416fce2ed6ad17ce"} Jul 09 17:13:44.128705 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:44.128670 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" event={"ID":"b7cc259f-0af4-4177-b4fc-328d68e08936","Type":"ContainerStarted","Data":"1ff9b6996c65daf1a119d27e22393da6ec1f61d7bc9e77fd3a270f81ca107b47"} Jul 09 17:13:44.147295 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:44.147244 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jgxgr" podStartSLOduration=1.784568282 podStartE2EDuration="15.147228849s" totalImagesPullingTime="13.362660567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:13:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:13:44.145737718 +0000 UTC m=+284.075183231" watchObservedRunningTime="2026-07-09 17:13:44.147228849 +0000 UTC m=+284.076674361" Jul 09 17:13:59.697696 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.697661 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln"] Jul 09 17:13:59.700226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.700210 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.703715 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.703692 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7d59w\"" Jul 09 17:13:59.703936 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.703920 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 17:13:59.703992 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.703974 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 17:13:59.704075 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.704026 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 17:13:59.704232 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.704216 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 17:13:59.726923 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.726894 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln"] Jul 09 17:13:59.842919 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.842882 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-manifests\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.843113 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.842967 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-tmp\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.843113 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.843048 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-webhook-cert\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.843113 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.843104 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r25d\" (UniqueName: \"kubernetes.io/projected/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-kube-api-access-9r25d\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.843262 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.843184 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.944618 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.944582 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-manifests\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.944618 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.944621 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-tmp\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.944860 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.944664 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-webhook-cert\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.944860 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.944693 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r25d\" (UniqueName: \"kubernetes.io/projected/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-kube-api-access-9r25d\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.944860 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.944750 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.945106 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.945073 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-manifests\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.947261 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.947234 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-webhook-cert\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.947479 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.947431 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-tmp\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.947627 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.947610 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:13:59.954927 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:13:59.954864 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r25d\" (UniqueName: \"kubernetes.io/projected/1d9dbdf0-9a28-4a2f-b110-2e767aeb57db-kube-api-access-9r25d\") pod \"opendatahub-operator-controller-manager-849f76587-4p6ln\" (UID: \"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:14:00.010475 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:00.010417 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:14:00.149881 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:00.149845 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln"] Jul 09 17:14:00.152822 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:14:00.152790 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d9dbdf0_9a28_4a2f_b110_2e767aeb57db.slice/crio-6078cb1c6713bcf90415b63c6edd5c19e7cd9e0918dc7aeffa81548bba35a052 WatchSource:0}: Error finding container 6078cb1c6713bcf90415b63c6edd5c19e7cd9e0918dc7aeffa81548bba35a052: Status 404 returned error can't find the container with id 6078cb1c6713bcf90415b63c6edd5c19e7cd9e0918dc7aeffa81548bba35a052 Jul 09 17:14:00.197833 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:00.197803 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" event={"ID":"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db","Type":"ContainerStarted","Data":"6078cb1c6713bcf90415b63c6edd5c19e7cd9e0918dc7aeffa81548bba35a052"} Jul 09 17:14:00.482647 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:00.482619 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:14:00.483123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:00.483101 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:14:00.487420 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:00.487404 2586 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 17:14:04.224615 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:04.224574 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" event={"ID":"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db","Type":"ContainerStarted","Data":"2c5a438622b41376707328bcda2debe35757ad960220209167e9b540fff03ca4"} Jul 09 17:14:05.230331 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:05.230291 2586 generic.go:356] "Generic (PLEG): container finished" podID="1d9dbdf0-9a28-4a2f-b110-2e767aeb57db" containerID="2c5a438622b41376707328bcda2debe35757ad960220209167e9b540fff03ca4" exitCode=0 Jul 09 17:14:05.230771 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:05.230373 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" event={"ID":"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db","Type":"ContainerDied","Data":"2c5a438622b41376707328bcda2debe35757ad960220209167e9b540fff03ca4"} Jul 09 17:14:05.230951 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:05.230935 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:14:06.237086 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:06.237047 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" event={"ID":"1d9dbdf0-9a28-4a2f-b110-2e767aeb57db","Type":"ContainerStarted","Data":"13321d521fb736767dee718b05b85a7469b20d9c565bd67fe379f1043bcd560a"} Jul 09 17:14:06.237484 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:06.237178 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:14:06.260814 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:06.260760 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" podStartSLOduration=2.417111401 podStartE2EDuration="7.260742133s" totalImagesPullingTime="3.691777969s" totalInitContainerRuntime="1.151852763s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:13:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:14:06.259393044 +0000 UTC m=+306.188838555" watchObservedRunningTime="2026-07-09 17:14:06.260742133 +0000 UTC m=+306.190187644" Jul 09 17:14:17.245325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:17.245289 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-4p6ln" Jul 09 17:14:19.707692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.707647 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9"] Jul 09 17:14:19.717705 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.717676 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.721735 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.721706 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5hpn8\"" Jul 09 17:14:19.721898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.721837 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 17:14:19.721977 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.721914 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 17:14:19.721977 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.721960 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 17:14:19.722225 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.722200 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9"] Jul 09 17:14:19.820704 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.820673 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ddc21b4e-fdc2-43cf-82ab-386404103633-metrics-cert\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.820872 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.820754 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trrpq\" (UniqueName: \"kubernetes.io/projected/ddc21b4e-fdc2-43cf-82ab-386404103633-kube-api-access-trrpq\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.820872 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.820782 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddc21b4e-fdc2-43cf-82ab-386404103633-cert\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.820872 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.820818 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ddc21b4e-fdc2-43cf-82ab-386404103633-manager-config\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.921511 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.921461 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ddc21b4e-fdc2-43cf-82ab-386404103633-manager-config\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.921723 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.921544 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ddc21b4e-fdc2-43cf-82ab-386404103633-metrics-cert\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.921723 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.921648 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trrpq\" (UniqueName: \"kubernetes.io/projected/ddc21b4e-fdc2-43cf-82ab-386404103633-kube-api-access-trrpq\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.921723 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.921691 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddc21b4e-fdc2-43cf-82ab-386404103633-cert\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.922150 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.922125 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ddc21b4e-fdc2-43cf-82ab-386404103633-manager-config\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.924053 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.924027 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ddc21b4e-fdc2-43cf-82ab-386404103633-metrics-cert\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.924053 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.924048 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ddc21b4e-fdc2-43cf-82ab-386404103633-cert\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:19.930211 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:19.930180 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trrpq\" (UniqueName: \"kubernetes.io/projected/ddc21b4e-fdc2-43cf-82ab-386404103633-kube-api-access-trrpq\") pod \"lws-controller-manager-59885db7cf-mxhq9\" (UID: \"ddc21b4e-fdc2-43cf-82ab-386404103633\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:20.028982 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:20.028941 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:20.168657 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:20.168632 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9"] Jul 09 17:14:20.171007 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:14:20.170973 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc21b4e_fdc2_43cf_82ab_386404103633.slice/crio-1ca324a4c5aae60b8367135598761b76c651b358a29bcc2042964c13fea64b1f WatchSource:0}: Error finding container 1ca324a4c5aae60b8367135598761b76c651b358a29bcc2042964c13fea64b1f: Status 404 returned error can't find the container with id 1ca324a4c5aae60b8367135598761b76c651b358a29bcc2042964c13fea64b1f Jul 09 17:14:20.309910 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:20.309822 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" event={"ID":"ddc21b4e-fdc2-43cf-82ab-386404103633","Type":"ContainerStarted","Data":"1ca324a4c5aae60b8367135598761b76c651b358a29bcc2042964c13fea64b1f"} Jul 09 17:14:22.321140 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:22.321103 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" event={"ID":"ddc21b4e-fdc2-43cf-82ab-386404103633","Type":"ContainerStarted","Data":"25fc7e8c5f39342f8d0548f3a4feb8006643b1ad8ae62c2e09168e1ce909d321"} Jul 09 17:14:22.321553 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:22.321170 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:22.339013 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:22.338952 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" podStartSLOduration=1.579204295 podStartE2EDuration="3.338932521s" totalImagesPullingTime="1.759728226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:14:22.337838666 +0000 UTC m=+322.267284182" watchObservedRunningTime="2026-07-09 17:14:22.338932521 +0000 UTC m=+322.268378033" Jul 09 17:14:27.373102 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.373067 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj"] Jul 09 17:14:27.377105 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.377081 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.380459 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.380417 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 17:14:27.380619 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.380425 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 17:14:27.380619 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.380583 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 17:14:27.380748 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.380733 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ccv2r\"" Jul 09 17:14:27.388365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.388334 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k"] Jul 09 17:14:27.392261 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.392232 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj"] Jul 09 17:14:27.392430 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.392328 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.395386 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.395362 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 17:14:27.395548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.395372 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 17:14:27.395548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.395372 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 17:14:27.395548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.395418 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hw57m\"" Jul 09 17:14:27.408012 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.407976 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k"] Jul 09 17:14:27.491923 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.491878 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnkzc\" (UniqueName: \"kubernetes.io/projected/c1939905-ac25-4557-b8f8-5a297af75629-kube-api-access-tnkzc\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.491923 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.491925 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.492144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.491952 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.492144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492009 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.492144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492038 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.492144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492070 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1939905-ac25-4557-b8f8-5a297af75629-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.492144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492086 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.492144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492105 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.492144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492126 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1939905-ac25-4557-b8f8-5a297af75629-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.492144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492141 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.492572 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492258 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmx45\" (UniqueName: \"kubernetes.io/projected/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-kube-api-access-dmx45\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.492572 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492302 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1939905-ac25-4557-b8f8-5a297af75629-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.492572 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492355 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.492572 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492403 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.492572 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492434 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.492572 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.492491 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593137 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593085 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593145 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593173 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593196 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593222 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnkzc\" (UniqueName: \"kubernetes.io/projected/c1939905-ac25-4557-b8f8-5a297af75629-kube-api-access-tnkzc\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.593342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593277 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593320 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.593694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593353 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.593694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593370 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.593694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593416 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1939905-ac25-4557-b8f8-5a297af75629-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.593694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593599 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593614 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593652 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593692 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1939905-ac25-4557-b8f8-5a297af75629-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.593999 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593722 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593999 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593799 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmx45\" (UniqueName: \"kubernetes.io/projected/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-kube-api-access-dmx45\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.593999 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.593833 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1939905-ac25-4557-b8f8-5a297af75629-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.594155 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.594015 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.594371 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.594338 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.594547 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.594474 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.594621 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.594551 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c1939905-ac25-4557-b8f8-5a297af75629-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.594818 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.594749 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.596176 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.596147 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.596324 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.596305 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1939905-ac25-4557-b8f8-5a297af75629-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.596416 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.596394 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.596548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.596525 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c1939905-ac25-4557-b8f8-5a297af75629-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.596827 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.596811 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.596879 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.596848 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.623915 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.623843 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnkzc\" (UniqueName: \"kubernetes.io/projected/c1939905-ac25-4557-b8f8-5a297af75629-kube-api-access-tnkzc\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.624293 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.624272 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmx45\" (UniqueName: \"kubernetes.io/projected/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-kube-api-access-dmx45\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.624293 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.624282 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678zdnfj\" (UID: \"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.626401 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.626374 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1939905-ac25-4557-b8f8-5a297af75629-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fss2k\" (UID: \"c1939905-ac25-4557-b8f8-5a297af75629\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.689296 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.689258 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:27.703975 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.703947 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:27.854357 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.854327 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj"] Jul 09 17:14:27.857027 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:14:27.856992 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ef2fdd_8b6e_4f32_8de2_61b9ee13b6d0.slice/crio-fe6256bb3a68a05c0084641040a2c9909da2642c40714ee961b850310e9e31b7 WatchSource:0}: Error finding container fe6256bb3a68a05c0084641040a2c9909da2642c40714ee961b850310e9e31b7: Status 404 returned error can't find the container with id fe6256bb3a68a05c0084641040a2c9909da2642c40714ee961b850310e9e31b7 Jul 09 17:14:27.873636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:27.873607 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k"] Jul 09 17:14:27.874967 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:14:27.874898 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1939905_ac25_4557_b8f8_5a297af75629.slice/crio-09eb5aa5a749082f07d3ba9c3fc0d9e5f0613889d0cdaf26712343f908b4e28c WatchSource:0}: Error finding container 09eb5aa5a749082f07d3ba9c3fc0d9e5f0613889d0cdaf26712343f908b4e28c: Status 404 returned error can't find the container with id 09eb5aa5a749082f07d3ba9c3fc0d9e5f0613889d0cdaf26712343f908b4e28c Jul 09 17:14:28.361498 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:28.361452 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" event={"ID":"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0","Type":"ContainerStarted","Data":"fe6256bb3a68a05c0084641040a2c9909da2642c40714ee961b850310e9e31b7"} Jul 09 17:14:28.363393 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:28.363354 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" event={"ID":"c1939905-ac25-4557-b8f8-5a297af75629","Type":"ContainerStarted","Data":"09eb5aa5a749082f07d3ba9c3fc0d9e5f0613889d0cdaf26712343f908b4e28c"} Jul 09 17:14:31.453494 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:31.453432 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:14:31.453788 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:31.453537 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:14:31.457133 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:31.457102 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:14:31.457219 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:31.457177 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:14:31.457219 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:31.457207 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:14:32.387191 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:32.387149 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" event={"ID":"c1939905-ac25-4557-b8f8-5a297af75629","Type":"ContainerStarted","Data":"9608a6cdde46d3099846a2785eb9a45180a21ff1329521e87527d56f137355bc"} Jul 09 17:14:32.387400 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:32.387355 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:32.388864 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:32.388834 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" event={"ID":"89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0","Type":"ContainerStarted","Data":"4fbd3335f3c8214668966f980eea284842a2813290c21c5cf20d79d3bb56d8ea"} Jul 09 17:14:32.389266 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:32.389245 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" Jul 09 17:14:32.418034 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:32.417980 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fss2k" podStartSLOduration=1.8419754780000002 podStartE2EDuration="5.417964236s" totalImagesPullingTime="3.575988758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:14:32.416252042 +0000 UTC m=+332.345697573" watchObservedRunningTime="2026-07-09 17:14:32.417964236 +0000 UTC m=+332.347409747" Jul 09 17:14:32.523167 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:32.523112 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" podStartSLOduration=1.92537029 podStartE2EDuration="5.523092747s" totalImagesPullingTime="3.597722457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:14:32.463558659 +0000 UTC m=+332.393004172" watchObservedRunningTime="2026-07-09 17:14:32.523092747 +0000 UTC m=+332.452538258" Jul 09 17:14:32.690373 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:32.690299 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:32.695082 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:32.695060 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:33.335483 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:33.335423 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-mxhq9" Jul 09 17:14:33.394469 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:33.394412 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:14:33.395469 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:14:33.395426 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678zdnfj" Jul 09 17:16:53.008554 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.008523 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw"] Jul 09 17:16:53.011187 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.011168 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.014347 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.014326 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5t5pq\"" Jul 09 17:16:53.023075 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.023051 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw"] Jul 09 17:16:53.067386 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067354 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.067573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067404 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17015692-b9b8-4cd2-b976-100fa0044291-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.067573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067454 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.067573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067499 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.067573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067557 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17015692-b9b8-4cd2-b976-100fa0044291-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.067729 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067579 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.067729 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067594 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17015692-b9b8-4cd2-b976-100fa0044291-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.067729 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067679 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.067729 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.067712 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4j9s\" (UniqueName: \"kubernetes.io/projected/17015692-b9b8-4cd2-b976-100fa0044291-kube-api-access-w4j9s\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169112 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169069 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169141 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17015692-b9b8-4cd2-b976-100fa0044291-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169165 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169182 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17015692-b9b8-4cd2-b976-100fa0044291-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169209 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169228 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4j9s\" (UniqueName: \"kubernetes.io/projected/17015692-b9b8-4cd2-b976-100fa0044291-kube-api-access-w4j9s\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169263 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169316 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17015692-b9b8-4cd2-b976-100fa0044291-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169352 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169601 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169769 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169627 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169967 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169889 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.169967 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.169910 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.170236 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.170213 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17015692-b9b8-4cd2-b976-100fa0044291-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.171609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.171587 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17015692-b9b8-4cd2-b976-100fa0044291-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.171746 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.171729 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17015692-b9b8-4cd2-b976-100fa0044291-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.177298 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.177273 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17015692-b9b8-4cd2-b976-100fa0044291-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.177454 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.177422 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4j9s\" (UniqueName: \"kubernetes.io/projected/17015692-b9b8-4cd2-b976-100fa0044291-kube-api-access-w4j9s\") pod \"maas-default-gateway-openshift-default-5896d4dd49-zpstw\" (UID: \"17015692-b9b8-4cd2-b976-100fa0044291\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.323124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.323033 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:53.457625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.457600 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw"] Jul 09 17:16:53.459474 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:16:53.459418 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17015692_b9b8_4cd2_b976_100fa0044291.slice/crio-fd4b90f53f3060507c599a133b83dfe164e03b63c5b98b289274635649cda583 WatchSource:0}: Error finding container fd4b90f53f3060507c599a133b83dfe164e03b63c5b98b289274635649cda583: Status 404 returned error can't find the container with id fd4b90f53f3060507c599a133b83dfe164e03b63c5b98b289274635649cda583 Jul 09 17:16:53.461704 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.461671 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:16:53.461796 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.461733 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:16:53.461796 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:53.461760 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:16:54.063087 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:54.063045 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" event={"ID":"17015692-b9b8-4cd2-b976-100fa0044291","Type":"ContainerStarted","Data":"9e61e73e0245d3cac0b6924207bd10892b0ed4683f73a2fbb3a1bf67faf79150"} Jul 09 17:16:54.063529 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:54.063113 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" event={"ID":"17015692-b9b8-4cd2-b976-100fa0044291","Type":"ContainerStarted","Data":"fd4b90f53f3060507c599a133b83dfe164e03b63c5b98b289274635649cda583"} Jul 09 17:16:54.086111 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:54.086062 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" podStartSLOduration=2.086043547 podStartE2EDuration="2.086043547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:16:54.083482357 +0000 UTC m=+474.012927870" watchObservedRunningTime="2026-07-09 17:16:54.086043547 +0000 UTC m=+474.015489058" Jul 09 17:16:54.323330 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:54.323224 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:54.328328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:54.328302 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:55.068326 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:55.068286 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:16:55.069384 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:16:55.069366 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:17:46.603591 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.603554 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79dbcf8674-4gcqg"] Jul 09 17:17:46.606246 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.606221 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:46.613184 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.613150 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 09 17:17:46.613334 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.613193 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 09 17:17:46.613334 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.613225 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vxpl9\"" Jul 09 17:17:46.617715 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.617695 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79dbcf8674-4gcqg"] Jul 09 17:17:46.770672 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.770633 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8d0f9da-43d9-4927-a19b-be9099d93c6d-webhook-cert\") pod \"maas-controller-79dbcf8674-4gcqg\" (UID: \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\") " pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:46.770870 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.770757 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47vxb\" (UniqueName: \"kubernetes.io/projected/f8d0f9da-43d9-4927-a19b-be9099d93c6d-kube-api-access-47vxb\") pod \"maas-controller-79dbcf8674-4gcqg\" (UID: \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\") " pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:46.871697 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.871593 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47vxb\" (UniqueName: \"kubernetes.io/projected/f8d0f9da-43d9-4927-a19b-be9099d93c6d-kube-api-access-47vxb\") pod \"maas-controller-79dbcf8674-4gcqg\" (UID: \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\") " pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:46.871697 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.871676 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8d0f9da-43d9-4927-a19b-be9099d93c6d-webhook-cert\") pod \"maas-controller-79dbcf8674-4gcqg\" (UID: \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\") " pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:46.874104 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.874077 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8d0f9da-43d9-4927-a19b-be9099d93c6d-webhook-cert\") pod \"maas-controller-79dbcf8674-4gcqg\" (UID: \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\") " pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:46.879304 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.879283 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47vxb\" (UniqueName: \"kubernetes.io/projected/f8d0f9da-43d9-4927-a19b-be9099d93c6d-kube-api-access-47vxb\") pod \"maas-controller-79dbcf8674-4gcqg\" (UID: \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\") " pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:46.925629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:46.925576 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:47.062230 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:47.062200 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79dbcf8674-4gcqg"] Jul 09 17:17:47.064614 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:17:47.064572 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d0f9da_43d9_4927_a19b_be9099d93c6d.slice/crio-d989b466f3dbdd3cff12a991af6fed53fa89eca8591df222ae774d92aa28e927 WatchSource:0}: Error finding container d989b466f3dbdd3cff12a991af6fed53fa89eca8591df222ae774d92aa28e927: Status 404 returned error can't find the container with id d989b466f3dbdd3cff12a991af6fed53fa89eca8591df222ae774d92aa28e927 Jul 09 17:17:47.321612 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:47.321576 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" event={"ID":"f8d0f9da-43d9-4927-a19b-be9099d93c6d","Type":"ContainerStarted","Data":"d989b466f3dbdd3cff12a991af6fed53fa89eca8591df222ae774d92aa28e927"} Jul 09 17:17:50.340579 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:50.340538 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" event={"ID":"f8d0f9da-43d9-4927-a19b-be9099d93c6d","Type":"ContainerStarted","Data":"56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a"} Jul 09 17:17:50.340995 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:50.340635 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:17:50.362019 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:17:50.361964 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" podStartSLOduration=2.143834919 podStartE2EDuration="4.361948584s" totalImagesPullingTime="2.218113665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:17:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:17:50.36062465 +0000 UTC m=+530.290070187" watchObservedRunningTime="2026-07-09 17:17:50.361948584 +0000 UTC m=+530.291394096" Jul 09 17:18:01.354108 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:01.354075 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:18:21.429577 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.429540 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-589b48b676-j4xhp"] Jul 09 17:18:21.433282 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.433260 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.438949 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.438926 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 17:18:21.439062 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.438981 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 17:18:21.440846 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.440827 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-cd4gx\"" Jul 09 17:18:21.440846 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.440845 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 09 17:18:21.444374 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.444312 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-589b48b676-j4xhp"] Jul 09 17:18:21.479872 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.479808 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bktk2\" (UniqueName: \"kubernetes.io/projected/d3427597-e9e5-462b-8ea0-d0fc18597d5a-kube-api-access-bktk2\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.480080 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.479904 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3427597-e9e5-462b-8ea0-d0fc18597d5a-openshift-service-ca\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.480080 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.479986 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3427597-e9e5-462b-8ea0-d0fc18597d5a-maas-api-tls\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.581026 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.580993 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3427597-e9e5-462b-8ea0-d0fc18597d5a-maas-api-tls\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.581205 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.581117 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bktk2\" (UniqueName: \"kubernetes.io/projected/d3427597-e9e5-462b-8ea0-d0fc18597d5a-kube-api-access-bktk2\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.581205 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.581157 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3427597-e9e5-462b-8ea0-d0fc18597d5a-openshift-service-ca\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.581906 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.581880 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3427597-e9e5-462b-8ea0-d0fc18597d5a-openshift-service-ca\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.584019 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.583989 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3427597-e9e5-462b-8ea0-d0fc18597d5a-maas-api-tls\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.592052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.592017 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bktk2\" (UniqueName: \"kubernetes.io/projected/d3427597-e9e5-462b-8ea0-d0fc18597d5a-kube-api-access-bktk2\") pod \"maas-api-589b48b676-j4xhp\" (UID: \"d3427597-e9e5-462b-8ea0-d0fc18597d5a\") " pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.744174 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.744129 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:21.941014 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:18:21.940982 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3427597_e9e5_462b_8ea0_d0fc18597d5a.slice/crio-5a3ca4be9baa41745281bd6062a0a506f270be1dbb538e973fbea775c161692f WatchSource:0}: Error finding container 5a3ca4be9baa41745281bd6062a0a506f270be1dbb538e973fbea775c161692f: Status 404 returned error can't find the container with id 5a3ca4be9baa41745281bd6062a0a506f270be1dbb538e973fbea775c161692f Jul 09 17:18:21.952854 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:21.952807 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-589b48b676-j4xhp"] Jul 09 17:18:22.176085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.176000 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zv5sg"] Jul 09 17:18:22.180261 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.180241 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:22.183604 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.183581 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 09 17:18:22.183775 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.183580 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vg8r8\"" Jul 09 17:18:22.191607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.191585 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zv5sg"] Jul 09 17:18:22.288486 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.288413 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z27sn\" (UniqueName: \"kubernetes.io/projected/4fbc7353-a95a-4d78-a47e-8f387cebb21c-kube-api-access-z27sn\") pod \"payload-processing-58f86c7798-zv5sg\" (UID: \"4fbc7353-a95a-4d78-a47e-8f387cebb21c\") " pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:22.288709 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.288580 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fbc7353-a95a-4d78-a47e-8f387cebb21c-plugins-config-volume\") pod \"payload-processing-58f86c7798-zv5sg\" (UID: \"4fbc7353-a95a-4d78-a47e-8f387cebb21c\") " pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:22.390016 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.389979 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z27sn\" (UniqueName: \"kubernetes.io/projected/4fbc7353-a95a-4d78-a47e-8f387cebb21c-kube-api-access-z27sn\") pod \"payload-processing-58f86c7798-zv5sg\" (UID: \"4fbc7353-a95a-4d78-a47e-8f387cebb21c\") " pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:22.390194 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.390041 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fbc7353-a95a-4d78-a47e-8f387cebb21c-plugins-config-volume\") pod \"payload-processing-58f86c7798-zv5sg\" (UID: \"4fbc7353-a95a-4d78-a47e-8f387cebb21c\") " pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:22.390739 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.390720 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fbc7353-a95a-4d78-a47e-8f387cebb21c-plugins-config-volume\") pod \"payload-processing-58f86c7798-zv5sg\" (UID: \"4fbc7353-a95a-4d78-a47e-8f387cebb21c\") " pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:22.399790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.399761 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z27sn\" (UniqueName: \"kubernetes.io/projected/4fbc7353-a95a-4d78-a47e-8f387cebb21c-kube-api-access-z27sn\") pod \"payload-processing-58f86c7798-zv5sg\" (UID: \"4fbc7353-a95a-4d78-a47e-8f387cebb21c\") " pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:22.492143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.492105 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:22.499009 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.498976 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" event={"ID":"d3427597-e9e5-462b-8ea0-d0fc18597d5a","Type":"ContainerStarted","Data":"5a3ca4be9baa41745281bd6062a0a506f270be1dbb538e973fbea775c161692f"} Jul 09 17:18:22.642518 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:22.642143 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-zv5sg"] Jul 09 17:18:23.509780 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:23.509740 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" event={"ID":"4fbc7353-a95a-4d78-a47e-8f387cebb21c","Type":"ContainerStarted","Data":"593e6556dbdddf991a99a775124e55cee3224d0e3227afeeca327fd8d6a908c7"} Jul 09 17:18:24.520460 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:24.520175 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" event={"ID":"4fbc7353-a95a-4d78-a47e-8f387cebb21c","Type":"ContainerStarted","Data":"c341389561629554f4e19f0e5c91fa4c7a02c4aa949ecf1fb2d0e470a71fa28c"} Jul 09 17:18:24.520460 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:24.520301 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:24.538791 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:24.538722 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" podStartSLOduration=0.976601909 podStartE2EDuration="2.538704756s" totalImagesPullingTime="1.562102847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:18:24.536560276 +0000 UTC m=+564.466005789" watchObservedRunningTime="2026-07-09 17:18:24.538704756 +0000 UTC m=+564.468150267" Jul 09 17:18:26.532928 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:26.532893 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" event={"ID":"d3427597-e9e5-462b-8ea0-d0fc18597d5a","Type":"ContainerStarted","Data":"7d49e1485891d36a01cdbada440e9d3e1ad58df66833cdb22f6366dee66c3995"} Jul 09 17:18:26.533325 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:26.532997 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:26.550380 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:26.550327 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" podStartSLOduration=1.867932461 podStartE2EDuration="5.550310963s" totalImagesPullingTime="3.682378502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:18:26.549474742 +0000 UTC m=+566.478920254" watchObservedRunningTime="2026-07-09 17:18:26.550310963 +0000 UTC m=+566.479756475" Jul 09 17:18:30.554380 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:30.554346 2586 generic.go:356] "Generic (PLEG): container finished" podID="17015692-b9b8-4cd2-b976-100fa0044291" containerID="9e61e73e0245d3cac0b6924207bd10892b0ed4683f73a2fbb3a1bf67faf79150" exitCode=0 Jul 09 17:18:30.554914 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:30.554426 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" event={"ID":"17015692-b9b8-4cd2-b976-100fa0044291","Type":"ContainerDied","Data":"9e61e73e0245d3cac0b6924207bd10892b0ed4683f73a2fbb3a1bf67faf79150"} Jul 09 17:18:30.555007 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:30.554991 2586 scope.go:117] "RemoveContainer" containerID="9e61e73e0245d3cac0b6924207bd10892b0ed4683f73a2fbb3a1bf67faf79150" Jul 09 17:18:30.556136 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:30.556105 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:18:30.556243 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:30.556190 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:18:30.556309 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:30.556275 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:18:31.561422 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:31.561385 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" event={"ID":"17015692-b9b8-4cd2-b976-100fa0044291","Type":"ContainerStarted","Data":"fccf2453d5a79eb97b9c0ab4949eacd5f7f93e6f824302681db4f4c55b401fdf"} Jul 09 17:18:32.324212 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:32.324173 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:18:32.324494 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:32.324469 2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-zpstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 09 17:18:32.324590 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:32.324542 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" podUID="17015692-b9b8-4cd2-b976-100fa0044291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 09 17:18:32.543309 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:32.543274 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-589b48b676-j4xhp" Jul 09 17:18:33.323546 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:33.323503 2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-zpstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 09 17:18:33.323955 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:33.323562 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" podUID="17015692-b9b8-4cd2-b976-100fa0044291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 09 17:18:34.324371 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:34.324328 2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-zpstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 09 17:18:34.324805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:34.324386 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" podUID="17015692-b9b8-4cd2-b976-100fa0044291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 09 17:18:35.323877 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:35.323826 2586 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-zpstw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 09 17:18:35.324065 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:35.323894 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" podUID="17015692-b9b8-4cd2-b976-100fa0044291" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 09 17:18:35.529354 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:35.529318 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-zv5sg" Jul 09 17:18:36.328003 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:36.327969 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:18:36.328362 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:36.328342 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:18:36.328953 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:36.328937 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-zpstw" Jul 09 17:18:57.376010 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.375924 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kf855"] Jul 09 17:18:57.379319 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.379294 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.382755 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.382728 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 17:18:57.384174 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.384151 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 17:18:57.384308 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.384247 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 17:18:57.384308 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.384284 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q9m66\"" Jul 09 17:18:57.390105 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.390041 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kf855"] Jul 09 17:18:57.454749 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.454711 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b544854-8484-4e55-af57-b7d0bc5f566b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.454941 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.454753 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.454941 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.454804 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.454941 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.454860 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.454941 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.454895 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs2mx\" (UniqueName: \"kubernetes.io/projected/7b544854-8484-4e55-af57-b7d0bc5f566b-kube-api-access-cs2mx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.455149 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.455023 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.556570 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.556532 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b544854-8484-4e55-af57-b7d0bc5f566b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.556570 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.556575 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.556894 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.556701 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.556894 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.556776 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.556894 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.556821 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs2mx\" (UniqueName: \"kubernetes.io/projected/7b544854-8484-4e55-af57-b7d0bc5f566b-kube-api-access-cs2mx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.557042 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.556905 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.557220 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.557192 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.557364 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.557228 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.557364 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.557318 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.559107 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.559084 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b544854-8484-4e55-af57-b7d0bc5f566b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.559515 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.559492 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b544854-8484-4e55-af57-b7d0bc5f566b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.565422 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.565402 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs2mx\" (UniqueName: \"kubernetes.io/projected/7b544854-8484-4e55-af57-b7d0bc5f566b-kube-api-access-cs2mx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-kf855\" (UID: \"7b544854-8484-4e55-af57-b7d0bc5f566b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.692651 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.692563 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:18:57.835241 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:57.835211 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-kf855"] Jul 09 17:18:57.837214 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:18:57.837191 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b544854_8484_4e55_af57_b7d0bc5f566b.slice/crio-2559868b7bf369992ed1438bec6b4da6b88674267018db8a87f458c8a5985699 WatchSource:0}: Error finding container 2559868b7bf369992ed1438bec6b4da6b88674267018db8a87f458c8a5985699: Status 404 returned error can't find the container with id 2559868b7bf369992ed1438bec6b4da6b88674267018db8a87f458c8a5985699 Jul 09 17:18:58.708354 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:58.708315 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" event={"ID":"7b544854-8484-4e55-af57-b7d0bc5f566b","Type":"ContainerStarted","Data":"2559868b7bf369992ed1438bec6b4da6b88674267018db8a87f458c8a5985699"} Jul 09 17:18:59.714970 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:59.714935 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" event={"ID":"7b544854-8484-4e55-af57-b7d0bc5f566b","Type":"ContainerStarted","Data":"4e67375aae85d4acc3a81522f24144c7c3d0a3709441e8d9d249c5469082db11"} Jul 09 17:18:59.736345 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:18:59.736294 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" podStartSLOduration=1.6712275540000001 podStartE2EDuration="2.736276458s" totalImagesPullingTime="1.065048904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:18:59.734562153 +0000 UTC m=+599.664007666" watchObservedRunningTime="2026-07-09 17:18:59.736276458 +0000 UTC m=+599.665721962" Jul 09 17:19:00.515631 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:00.515600 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:19:00.515936 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:00.515613 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:19:07.693253 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:07.693206 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:19:07.693693 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:07.693295 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:19:07.705867 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:07.705828 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:19:07.763634 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:07.763600 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-kf855" Jul 09 17:19:09.494645 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.494608 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps"] Jul 09 17:19:09.498100 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.498078 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.501695 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.501678 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 17:19:09.507643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.507621 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps"] Jul 09 17:19:09.584551 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.584511 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.584721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.584556 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl7rb\" (UniqueName: \"kubernetes.io/projected/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-kube-api-access-hl7rb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.584721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.584603 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.584721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.584623 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.584721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.584675 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.584721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.584708 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.685745 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.685703 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.685924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.685750 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl7rb\" (UniqueName: \"kubernetes.io/projected/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-kube-api-access-hl7rb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.685924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.685792 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.685924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.685811 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.685924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.685859 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.685924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.685903 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.686288 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.686250 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.686415 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.686258 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.686415 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.686302 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.688026 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.688007 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.688377 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.688357 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.694746 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.694718 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl7rb\" (UniqueName: \"kubernetes.io/projected/859fc1ab-c51c-4f96-ae1f-1ee8633de13f-kube-api-access-hl7rb\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps\" (UID: \"859fc1ab-c51c-4f96-ae1f-1ee8633de13f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.809075 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.808975 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:09.947090 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.947064 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps"] Jul 09 17:19:09.949083 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:19:09.949051 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859fc1ab_c51c_4f96_ae1f_1ee8633de13f.slice/crio-32d7d35cf25075950181807e97780bc0a15b25a0b427f89df7844edd6037b0f2 WatchSource:0}: Error finding container 32d7d35cf25075950181807e97780bc0a15b25a0b427f89df7844edd6037b0f2: Status 404 returned error can't find the container with id 32d7d35cf25075950181807e97780bc0a15b25a0b427f89df7844edd6037b0f2 Jul 09 17:19:09.950426 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:09.950410 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:19:10.769141 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:10.769098 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" event={"ID":"859fc1ab-c51c-4f96-ae1f-1ee8633de13f","Type":"ContainerStarted","Data":"fdc66c4bbc526882f766f4d8ca68af5e832321b4b3f81fb5963b69ca223ac074"} Jul 09 17:19:10.769589 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:10.769163 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" event={"ID":"859fc1ab-c51c-4f96-ae1f-1ee8633de13f","Type":"ContainerStarted","Data":"32d7d35cf25075950181807e97780bc0a15b25a0b427f89df7844edd6037b0f2"} Jul 09 17:19:10.787083 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:10.787029 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" podStartSLOduration=1.5827385729999999 podStartE2EDuration="1.787009429s" totalImagesPullingTime="204.270856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:19:10.786066622 +0000 UTC m=+610.715512146" watchObservedRunningTime="2026-07-09 17:19:10.787009429 +0000 UTC m=+610.716454942" Jul 09 17:19:19.809331 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:19.809298 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:19.809737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:19.809405 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:19.822457 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:19.822403 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:19:19.836508 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:19:19.836481 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps" Jul 09 17:21:44.064904 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.064834 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79dbcf8674-4gcqg"] Jul 09 17:21:44.065645 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.065247 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" podUID="f8d0f9da-43d9-4927-a19b-be9099d93c6d" containerName="manager" containerID="cri-o://56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a" gracePeriod=30 Jul 09 17:21:44.319699 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.319627 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:21:44.434909 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.434879 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47vxb\" (UniqueName: \"kubernetes.io/projected/f8d0f9da-43d9-4927-a19b-be9099d93c6d-kube-api-access-47vxb\") pod \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\" (UID: \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\") " Jul 09 17:21:44.435087 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.434931 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8d0f9da-43d9-4927-a19b-be9099d93c6d-webhook-cert\") pod \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\" (UID: \"f8d0f9da-43d9-4927-a19b-be9099d93c6d\") " Jul 09 17:21:44.437096 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.437062 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d0f9da-43d9-4927-a19b-be9099d93c6d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f8d0f9da-43d9-4927-a19b-be9099d93c6d" (UID: "f8d0f9da-43d9-4927-a19b-be9099d93c6d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:21:44.437096 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.437084 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d0f9da-43d9-4927-a19b-be9099d93c6d-kube-api-access-47vxb" (OuterVolumeSpecName: "kube-api-access-47vxb") pod "f8d0f9da-43d9-4927-a19b-be9099d93c6d" (UID: "f8d0f9da-43d9-4927-a19b-be9099d93c6d"). InnerVolumeSpecName "kube-api-access-47vxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:21:44.535920 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.535881 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47vxb\" (UniqueName: \"kubernetes.io/projected/f8d0f9da-43d9-4927-a19b-be9099d93c6d-kube-api-access-47vxb\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:21:44.535920 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.535914 2586 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8d0f9da-43d9-4927-a19b-be9099d93c6d-webhook-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:21:44.572318 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.572236 2586 generic.go:356] "Generic (PLEG): container finished" podID="f8d0f9da-43d9-4927-a19b-be9099d93c6d" containerID="56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a" exitCode=0 Jul 09 17:21:44.572318 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.572274 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" event={"ID":"f8d0f9da-43d9-4927-a19b-be9099d93c6d","Type":"ContainerDied","Data":"56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a"} Jul 09 17:21:44.572318 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.572284 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" Jul 09 17:21:44.572318 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.572314 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79dbcf8674-4gcqg" event={"ID":"f8d0f9da-43d9-4927-a19b-be9099d93c6d","Type":"ContainerDied","Data":"d989b466f3dbdd3cff12a991af6fed53fa89eca8591df222ae774d92aa28e927"} Jul 09 17:21:44.572609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.572336 2586 scope.go:117] "RemoveContainer" containerID="56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a" Jul 09 17:21:44.581584 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.581567 2586 scope.go:117] "RemoveContainer" containerID="56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a" Jul 09 17:21:44.581859 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:21:44.581839 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a\": container with ID starting with 56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a not found: ID does not exist" containerID="56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a" Jul 09 17:21:44.581915 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.581869 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a"} err="failed to get container status \"56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a\": rpc error: code = NotFound desc = could not find container \"56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a\": container with ID starting with 56e9dec6d9d7d7264cf25e5d02d1a69c624b49f9aef412a8b003f923950ef20a not found: ID does not exist" Jul 09 17:21:44.594363 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.594338 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79dbcf8674-4gcqg"] Jul 09 17:21:44.598865 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.598844 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79dbcf8674-4gcqg"] Jul 09 17:21:44.671435 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:21:44.671403 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8d0f9da-43d9-4927-a19b-be9099d93c6d" path="/var/lib/kubelet/pods/f8d0f9da-43d9-4927-a19b-be9099d93c6d/volumes" Jul 09 17:24:00.548935 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:24:00.548901 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:24:00.551068 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:24:00.551039 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:29:00.581547 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:29:00.581520 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:29:00.585792 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:29:00.585770 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:33:55.817693 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.817658 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf"] Jul 09 17:33:55.818334 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.818314 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8d0f9da-43d9-4927-a19b-be9099d93c6d" containerName="manager" Jul 09 17:33:55.818416 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.818338 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8d0f9da-43d9-4927-a19b-be9099d93c6d" containerName="manager" Jul 09 17:33:55.818514 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.818484 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8d0f9da-43d9-4927-a19b-be9099d93c6d" containerName="manager" Jul 09 17:33:55.820575 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.820555 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.823722 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.823702 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-76fddcd7-openshift-default-dockercfg-bd887\"" Jul 09 17:33:55.840817 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.840789 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf"] Jul 09 17:33:55.970100 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970064 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-socket\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.970100 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970103 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-certs\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.970344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970131 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-envoy\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.970344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970157 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flkpd\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-kube-api-access-flkpd\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.970344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970239 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-credential-socket\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.970344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970318 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4677ab-3902-4433-b093-5c61d77f003f-istio-podinfo\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.970344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970340 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4677ab-3902-4433-b093-5c61d77f003f-istiod-ca-cert\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.970569 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970357 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-istio-token\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:55.970569 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:55.970514 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-data\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.071914 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.071827 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-credential-socket\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.071914 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.071887 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4677ab-3902-4433-b093-5c61d77f003f-istio-podinfo\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.071914 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.071908 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4677ab-3902-4433-b093-5c61d77f003f-istiod-ca-cert\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.071927 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-istio-token\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072082 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-data\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072212 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072170 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-socket\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072212 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072198 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-certs\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072310 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072230 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-envoy\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072310 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072269 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-credential-socket\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072325 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flkpd\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-kube-api-access-flkpd\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072510 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072477 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-data\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072606 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072586 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-socket\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072606 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072597 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-certs\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.072836 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.072716 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4677ab-3902-4433-b093-5c61d77f003f-istiod-ca-cert\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.074460 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.074426 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-envoy\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.074575 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.074537 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4677ab-3902-4433-b093-5c61d77f003f-istio-podinfo\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.088171 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.088144 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-istio-token\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.088286 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.088226 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flkpd\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-kube-api-access-flkpd\") pod \"e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.131273 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.131236 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:56.271395 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.271368 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf"] Jul 09 17:33:56.273222 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:33:56.273192 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4677ab_3902_4433_b093_5c61d77f003f.slice/crio-6b17d0012508ef6253288878ccd513f9b51bddd73f34c52eeb0fab5cbf0ee8b9 WatchSource:0}: Error finding container 6b17d0012508ef6253288878ccd513f9b51bddd73f34c52eeb0fab5cbf0ee8b9: Status 404 returned error can't find the container with id 6b17d0012508ef6253288878ccd513f9b51bddd73f34c52eeb0fab5cbf0ee8b9 Jul 09 17:33:56.275402 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.275368 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:33:56.275481 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.275463 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:33:56.275524 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:56.275499 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:33:57.031875 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.031839 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6"] Jul 09 17:33:57.034356 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.034339 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.037599 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.037580 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-76fddcd7-serving-cert\"" Jul 09 17:33:57.046551 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.046529 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6"] Jul 09 17:33:57.182868 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.182832 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrxf8\" (UniqueName: \"kubernetes.io/projected/dcfacefc-0c05-4fed-98ba-934a879f62da-kube-api-access-zrxf8\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.183038 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.182890 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfacefc-0c05-4fed-98ba-934a879f62da-openshift-service-ca\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.183038 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.182972 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.272500 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.272465 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" event={"ID":"cd4677ab-3902-4433-b093-5c61d77f003f","Type":"ContainerStarted","Data":"5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb"} Jul 09 17:33:57.272668 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.272528 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" event={"ID":"cd4677ab-3902-4433-b093-5c61d77f003f","Type":"ContainerStarted","Data":"6b17d0012508ef6253288878ccd513f9b51bddd73f34c52eeb0fab5cbf0ee8b9"} Jul 09 17:33:57.284096 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.284025 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrxf8\" (UniqueName: \"kubernetes.io/projected/dcfacefc-0c05-4fed-98ba-934a879f62da-kube-api-access-zrxf8\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.284096 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.284087 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfacefc-0c05-4fed-98ba-934a879f62da-openshift-service-ca\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.284246 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.284121 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.284282 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:33:57.284252 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-serving-cert: secret "maas-api-e2e-ait-76fddcd7-serving-cert" not found Jul 09 17:33:57.284321 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:33:57.284308 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls podName:dcfacefc-0c05-4fed-98ba-934a879f62da nodeName:}" failed. No retries permitted until 2026-07-09 17:33:57.784289344 +0000 UTC m=+1497.713734836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls") pod "maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" (UID: "dcfacefc-0c05-4fed-98ba-934a879f62da") : secret "maas-api-e2e-ait-76fddcd7-serving-cert" not found Jul 09 17:33:57.284738 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.284719 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfacefc-0c05-4fed-98ba-934a879f62da-openshift-service-ca\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.293467 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.293406 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" podStartSLOduration=2.293391345 podStartE2EDuration="2.293391345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:33:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:33:57.291954172 +0000 UTC m=+1497.221399686" watchObservedRunningTime="2026-07-09 17:33:57.293391345 +0000 UTC m=+1497.222836857" Jul 09 17:33:57.295739 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.295717 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrxf8\" (UniqueName: \"kubernetes.io/projected/dcfacefc-0c05-4fed-98ba-934a879f62da-kube-api-access-zrxf8\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.789789 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.789749 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.792306 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.792272 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls\") pod \"maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:57.946055 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:57.946018 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:33:58.080871 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:58.080836 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6"] Jul 09 17:33:58.082097 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:33:58.082068 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfacefc_0c05_4fed_98ba_934a879f62da.slice/crio-d94d24457b6d1e9476183b922630843643e65c2fa7ca41b8c430809934aea3b6 WatchSource:0}: Error finding container d94d24457b6d1e9476183b922630843643e65c2fa7ca41b8c430809934aea3b6: Status 404 returned error can't find the container with id d94d24457b6d1e9476183b922630843643e65c2fa7ca41b8c430809934aea3b6 Jul 09 17:33:58.083567 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:58.083550 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:33:58.131431 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:58.131398 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:58.136067 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:58.136043 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:58.278292 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:58.278253 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" event={"ID":"dcfacefc-0c05-4fed-98ba-934a879f62da","Type":"ContainerStarted","Data":"d94d24457b6d1e9476183b922630843643e65c2fa7ca41b8c430809934aea3b6"} Jul 09 17:33:58.278586 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:58.278568 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:58.279467 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:58.279425 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:33:59.285170 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:59.285131 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" event={"ID":"dcfacefc-0c05-4fed-98ba-934a879f62da","Type":"ContainerStarted","Data":"41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe"} Jul 09 17:33:59.302357 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:33:59.302297 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" podStartSLOduration=1.5786332010000002 podStartE2EDuration="2.302279841s" totalImagesPullingTime="723.64664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:33:59.300377402 +0000 UTC m=+1499.229822918" watchObservedRunningTime="2026-07-09 17:33:59.302279841 +0000 UTC m=+1499.231725352" Jul 09 17:34:00.290638 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:00.290599 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:34:00.613285 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:00.613112 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:34:00.627782 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:00.618530 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:34:02.239861 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.239827 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf"] Jul 09 17:34:02.240287 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.240003 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" podUID="cd4677ab-3902-4433-b093-5c61d77f003f" containerName="istio-proxy" containerID="cri-o://5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb" gracePeriod=30 Jul 09 17:34:02.411120 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.411080 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6"] Jul 09 17:34:02.411299 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.411277 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" podUID="dcfacefc-0c05-4fed-98ba-934a879f62da" containerName="maas-api" containerID="cri-o://41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe" gracePeriod=30 Jul 09 17:34:02.703267 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.703231 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:34:02.844067 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.843980 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfacefc-0c05-4fed-98ba-934a879f62da-openshift-service-ca\") pod \"dcfacefc-0c05-4fed-98ba-934a879f62da\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " Jul 09 17:34:02.844067 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.844030 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls\") pod \"dcfacefc-0c05-4fed-98ba-934a879f62da\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " Jul 09 17:34:02.844249 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.844202 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrxf8\" (UniqueName: \"kubernetes.io/projected/dcfacefc-0c05-4fed-98ba-934a879f62da-kube-api-access-zrxf8\") pod \"dcfacefc-0c05-4fed-98ba-934a879f62da\" (UID: \"dcfacefc-0c05-4fed-98ba-934a879f62da\") " Jul 09 17:34:02.844364 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.844338 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcfacefc-0c05-4fed-98ba-934a879f62da-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dcfacefc-0c05-4fed-98ba-934a879f62da" (UID: "dcfacefc-0c05-4fed-98ba-934a879f62da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:34:02.844622 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.844599 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcfacefc-0c05-4fed-98ba-934a879f62da-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:02.846245 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.846220 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dcfacefc-0c05-4fed-98ba-934a879f62da" (UID: "dcfacefc-0c05-4fed-98ba-934a879f62da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:34:02.846245 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.846232 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcfacefc-0c05-4fed-98ba-934a879f62da-kube-api-access-zrxf8" (OuterVolumeSpecName: "kube-api-access-zrxf8") pod "dcfacefc-0c05-4fed-98ba-934a879f62da" (UID: "dcfacefc-0c05-4fed-98ba-934a879f62da"). InnerVolumeSpecName "kube-api-access-zrxf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:02.945332 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.945298 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcfacefc-0c05-4fed-98ba-934a879f62da-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:02.945332 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:02.945330 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrxf8\" (UniqueName: \"kubernetes.io/projected/dcfacefc-0c05-4fed-98ba-934a879f62da-kube-api-access-zrxf8\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:03.310066 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.310023 2586 generic.go:356] "Generic (PLEG): container finished" podID="dcfacefc-0c05-4fed-98ba-934a879f62da" containerID="41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe" exitCode=0 Jul 09 17:34:03.310066 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.310071 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" Jul 09 17:34:03.310564 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.310099 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" event={"ID":"dcfacefc-0c05-4fed-98ba-934a879f62da","Type":"ContainerDied","Data":"41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe"} Jul 09 17:34:03.310564 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.310153 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6" event={"ID":"dcfacefc-0c05-4fed-98ba-934a879f62da","Type":"ContainerDied","Data":"d94d24457b6d1e9476183b922630843643e65c2fa7ca41b8c430809934aea3b6"} Jul 09 17:34:03.310564 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.310178 2586 scope.go:117] "RemoveContainer" containerID="41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe" Jul 09 17:34:03.319518 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.319475 2586 scope.go:117] "RemoveContainer" containerID="41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe" Jul 09 17:34:03.319852 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:03.319831 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe\": container with ID starting with 41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe not found: ID does not exist" containerID="41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe" Jul 09 17:34:03.319911 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.319865 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe"} err="failed to get container status \"41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe\": rpc error: code = NotFound desc = could not find container \"41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe\": container with ID starting with 41055d541f0fb96f27c99b0e351d63792133e198d55f226e3245ede9edc6fabe not found: ID does not exist" Jul 09 17:34:03.332775 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.332743 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6"] Jul 09 17:34:03.336676 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:03.336646 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-76fddcd7-7dd4b9d9cb-djmv6"] Jul 09 17:34:04.671896 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:04.671857 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcfacefc-0c05-4fed-98ba-934a879f62da" path="/var/lib/kubelet/pods/dcfacefc-0c05-4fed-98ba-934a879f62da/volumes" Jul 09 17:34:08.324807 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.324769 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq"] Jul 09 17:34:08.325463 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.325426 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcfacefc-0c05-4fed-98ba-934a879f62da" containerName="maas-api" Jul 09 17:34:08.325463 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.325461 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcfacefc-0c05-4fed-98ba-934a879f62da" containerName="maas-api" Jul 09 17:34:08.325583 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.325566 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcfacefc-0c05-4fed-98ba-934a879f62da" containerName="maas-api" Jul 09 17:34:08.329288 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.329262 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.332499 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.332471 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2e22ec17-openshift-default-dockercfg-4v9ff\"" Jul 09 17:34:08.343074 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.343045 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq"] Jul 09 17:34:08.415235 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415195 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-istio-token\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.415235 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415241 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7311ac88-0260-437b-a074-6e53896ed8d3-istiod-ca-cert\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.415493 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415273 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-envoy\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.415493 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415338 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-credential-socket\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.415493 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415374 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-data\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.415605 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415511 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-socket\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.415605 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415545 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46g29\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-kube-api-access-46g29\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.415673 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415615 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7311ac88-0260-437b-a074-6e53896ed8d3-istio-podinfo\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.415673 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.415644 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-certs\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516635 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-credential-socket\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516678 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516680 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-data\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516735 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-socket\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516758 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46g29\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-kube-api-access-46g29\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516799 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7311ac88-0260-437b-a074-6e53896ed8d3-istio-podinfo\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516820 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-certs\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516864 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-istio-token\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516896 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7311ac88-0260-437b-a074-6e53896ed8d3-istiod-ca-cert\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.516952 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.516934 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-envoy\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.517314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.517160 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-credential-socket\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.517314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.517251 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-socket\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.517497 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.517468 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-data\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.517654 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.517605 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-certs\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.517946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.517919 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7311ac88-0260-437b-a074-6e53896ed8d3-istiod-ca-cert\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.519310 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.519287 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-envoy\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.519690 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.519664 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7311ac88-0260-437b-a074-6e53896ed8d3-istio-podinfo\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.525311 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.525284 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-istio-token\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.525468 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.525353 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46g29\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-kube-api-access-46g29\") pod \"e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.645474 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.645344 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:08.789827 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.789787 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq"] Jul 09 17:34:08.790958 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:34:08.790928 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7311ac88_0260_437b_a074_6e53896ed8d3.slice/crio-1c658c09bd343550e39a77e9f87417f5bc741e5fccff54474f58329cb19c96f5 WatchSource:0}: Error finding container 1c658c09bd343550e39a77e9f87417f5bc741e5fccff54474f58329cb19c96f5: Status 404 returned error can't find the container with id 1c658c09bd343550e39a77e9f87417f5bc741e5fccff54474f58329cb19c96f5 Jul 09 17:34:08.793276 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.793244 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:08.793365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.793310 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:08.793365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:08.793337 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:09.092272 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.092242 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:34:09.123759 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.123726 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-istio-token\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.123946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.123824 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-socket\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.123946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.123855 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4677ab-3902-4433-b093-5c61d77f003f-istio-podinfo\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.123946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.123893 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-certs\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.123946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.123922 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-data\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.124159 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.123954 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-credential-socket\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.124159 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.124034 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4677ab-3902-4433-b093-5c61d77f003f-istiod-ca-cert\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.124159 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.124088 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flkpd\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-kube-api-access-flkpd\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.124382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.124239 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:09.124382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.124299 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-envoy\") pod \"cd4677ab-3902-4433-b093-5c61d77f003f\" (UID: \"cd4677ab-3902-4433-b093-5c61d77f003f\") " Jul 09 17:34:09.124636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.124601 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:09.124795 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.124777 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.124875 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.124803 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.124937 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.124891 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:09.125231 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.125190 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd4677ab-3902-4433-b093-5c61d77f003f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:34:09.125535 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.125507 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:09.126495 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.126464 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd4677ab-3902-4433-b093-5c61d77f003f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:34:09.126619 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.126595 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-kube-api-access-flkpd" (OuterVolumeSpecName: "kube-api-access-flkpd") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "kube-api-access-flkpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:09.126619 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.126602 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:09.127149 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.127119 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd4677ab-3902-4433-b093-5c61d77f003f" (UID: "cd4677ab-3902-4433-b093-5c61d77f003f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:09.226023 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.225991 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.226023 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.226024 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.226023 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.226033 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4677ab-3902-4433-b093-5c61d77f003f-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.226269 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.226041 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flkpd\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-kube-api-access-flkpd\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.226269 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.226050 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4677ab-3902-4433-b093-5c61d77f003f-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.226269 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.226058 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4677ab-3902-4433-b093-5c61d77f003f-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.226269 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.226066 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4677ab-3902-4433-b093-5c61d77f003f-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:09.352736 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.352690 2586 generic.go:356] "Generic (PLEG): container finished" podID="cd4677ab-3902-4433-b093-5c61d77f003f" containerID="5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb" exitCode=0 Jul 09 17:34:09.353200 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.352759 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" Jul 09 17:34:09.353200 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.352775 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" event={"ID":"cd4677ab-3902-4433-b093-5c61d77f003f","Type":"ContainerDied","Data":"5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb"} Jul 09 17:34:09.353200 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.352880 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf" event={"ID":"cd4677ab-3902-4433-b093-5c61d77f003f","Type":"ContainerDied","Data":"6b17d0012508ef6253288878ccd513f9b51bddd73f34c52eeb0fab5cbf0ee8b9"} Jul 09 17:34:09.353200 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.352920 2586 scope.go:117] "RemoveContainer" containerID="5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb" Jul 09 17:34:09.354823 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.354796 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" event={"ID":"7311ac88-0260-437b-a074-6e53896ed8d3","Type":"ContainerStarted","Data":"19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54"} Jul 09 17:34:09.354973 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.354847 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" event={"ID":"7311ac88-0260-437b-a074-6e53896ed8d3","Type":"ContainerStarted","Data":"1c658c09bd343550e39a77e9f87417f5bc741e5fccff54474f58329cb19c96f5"} Jul 09 17:34:09.363319 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.363271 2586 scope.go:117] "RemoveContainer" containerID="5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb" Jul 09 17:34:09.363658 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:09.363636 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb\": container with ID starting with 5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb not found: ID does not exist" containerID="5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb" Jul 09 17:34:09.363708 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.363669 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb"} err="failed to get container status \"5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb\": rpc error: code = NotFound desc = could not find container \"5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb\": container with ID starting with 5476ad384bce80dfd9e0283b3768161f0e78fe468962d649edb338f9e86113cb not found: ID does not exist" Jul 09 17:34:09.377158 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.377109 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" podStartSLOduration=1.37709382 podStartE2EDuration="1.37709382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:34:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:34:09.37494996 +0000 UTC m=+1509.304395489" watchObservedRunningTime="2026-07-09 17:34:09.37709382 +0000 UTC m=+1509.306539332" Jul 09 17:34:09.389596 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.389563 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf"] Jul 09 17:34:09.395687 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.395658 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-76fddcd7-openshift-default-56d7b668fc-wpcjf"] Jul 09 17:34:09.646514 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.646468 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:09.651412 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.651384 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:09.776607 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.776513 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq"] Jul 09 17:34:09.777091 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.777064 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd4677ab-3902-4433-b093-5c61d77f003f" containerName="istio-proxy" Jul 09 17:34:09.777091 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.777089 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd4677ab-3902-4433-b093-5c61d77f003f" containerName="istio-proxy" Jul 09 17:34:09.777279 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.777173 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd4677ab-3902-4433-b093-5c61d77f003f" containerName="istio-proxy" Jul 09 17:34:09.779857 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.779838 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:09.782964 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.782945 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2e22ec17-serving-cert\"" Jul 09 17:34:09.796546 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.796518 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq"] Jul 09 17:34:09.831936 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.831906 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:09.832110 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.831949 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-openshift-service-ca\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:09.832110 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.832059 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tptq6\" (UniqueName: \"kubernetes.io/projected/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-kube-api-access-tptq6\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:09.933700 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.933670 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:09.933888 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.933717 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-openshift-service-ca\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:09.933888 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.933781 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tptq6\" (UniqueName: \"kubernetes.io/projected/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-kube-api-access-tptq6\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:09.933888 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:09.933818 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-serving-cert: secret "maas-api-e2e-ait-2e22ec17-serving-cert" not found Jul 09 17:34:09.934000 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:09.933891 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls podName:7da0f4c8-83ff-4a6f-8946-ef4988bba28a nodeName:}" failed. No retries permitted until 2026-07-09 17:34:10.433871822 +0000 UTC m=+1510.363317319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls") pod "maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" (UID: "7da0f4c8-83ff-4a6f-8946-ef4988bba28a") : secret "maas-api-e2e-ait-2e22ec17-serving-cert" not found Jul 09 17:34:09.934419 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.934398 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-openshift-service-ca\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:09.945192 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:09.945158 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tptq6\" (UniqueName: \"kubernetes.io/projected/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-kube-api-access-tptq6\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:10.360581 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:10.360549 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:10.361578 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:10.361555 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:10.439790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:10.439753 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:10.442098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:10.442072 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls\") pod \"maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:10.671954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:10.671868 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd4677ab-3902-4433-b093-5c61d77f003f" path="/var/lib/kubelet/pods/cd4677ab-3902-4433-b093-5c61d77f003f/volumes" Jul 09 17:34:10.690818 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:10.690785 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:10.845656 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:10.845625 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq"] Jul 09 17:34:10.846729 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:34:10.846699 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da0f4c8_83ff_4a6f_8946_ef4988bba28a.slice/crio-056098ced9160caa1a499866aa8ba339a432b71866f71cf82cdde30cb7fecfef WatchSource:0}: Error finding container 056098ced9160caa1a499866aa8ba339a432b71866f71cf82cdde30cb7fecfef: Status 404 returned error can't find the container with id 056098ced9160caa1a499866aa8ba339a432b71866f71cf82cdde30cb7fecfef Jul 09 17:34:11.367133 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:11.367092 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" event={"ID":"7da0f4c8-83ff-4a6f-8946-ef4988bba28a","Type":"ContainerStarted","Data":"ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503"} Jul 09 17:34:11.367567 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:11.367156 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:11.367567 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:11.367175 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" event={"ID":"7da0f4c8-83ff-4a6f-8946-ef4988bba28a","Type":"ContainerStarted","Data":"056098ced9160caa1a499866aa8ba339a432b71866f71cf82cdde30cb7fecfef"} Jul 09 17:34:11.388285 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:11.388224 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" podStartSLOduration=2.04022793 podStartE2EDuration="2.388201115s" totalImagesPullingTime="347.973185ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:34:11.385897555 +0000 UTC m=+1511.315343087" watchObservedRunningTime="2026-07-09 17:34:11.388201115 +0000 UTC m=+1511.317646628" Jul 09 17:34:14.952282 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:14.952252 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq"] Jul 09 17:34:14.952819 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:14.952524 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" podUID="7da0f4c8-83ff-4a6f-8946-ef4988bba28a" containerName="maas-api" containerID="cri-o://ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503" gracePeriod=30 Jul 09 17:34:14.975122 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:14.975082 2586 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da0f4c8_83ff_4a6f_8946_ef4988bba28a.slice/crio-conmon-ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503.scope\": RecentStats: unable to find data in memory cache]" Jul 09 17:34:14.975266 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:14.975084 2586 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da0f4c8_83ff_4a6f_8946_ef4988bba28a.slice/crio-ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503.scope\": RecentStats: unable to find data in memory cache]" Jul 09 17:34:15.256011 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.255986 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:15.406340 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.406307 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tptq6\" (UniqueName: \"kubernetes.io/projected/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-kube-api-access-tptq6\") pod \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " Jul 09 17:34:15.406582 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.406386 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls\") pod \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " Jul 09 17:34:15.406582 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.406493 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-openshift-service-ca\") pod \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\" (UID: \"7da0f4c8-83ff-4a6f-8946-ef4988bba28a\") " Jul 09 17:34:15.406923 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.406887 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7da0f4c8-83ff-4a6f-8946-ef4988bba28a" (UID: "7da0f4c8-83ff-4a6f-8946-ef4988bba28a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:34:15.406923 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.406914 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" event={"ID":"7da0f4c8-83ff-4a6f-8946-ef4988bba28a","Type":"ContainerDied","Data":"ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503"} Jul 09 17:34:15.407073 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.406891 2586 generic.go:356] "Generic (PLEG): container finished" podID="7da0f4c8-83ff-4a6f-8946-ef4988bba28a" containerID="ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503" exitCode=0 Jul 09 17:34:15.407073 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.406968 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" event={"ID":"7da0f4c8-83ff-4a6f-8946-ef4988bba28a","Type":"ContainerDied","Data":"056098ced9160caa1a499866aa8ba339a432b71866f71cf82cdde30cb7fecfef"} Jul 09 17:34:15.407073 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.406997 2586 scope.go:117] "RemoveContainer" containerID="ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503" Jul 09 17:34:15.407073 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.407019 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq" Jul 09 17:34:15.408771 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.408744 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-kube-api-access-tptq6" (OuterVolumeSpecName: "kube-api-access-tptq6") pod "7da0f4c8-83ff-4a6f-8946-ef4988bba28a" (UID: "7da0f4c8-83ff-4a6f-8946-ef4988bba28a"). InnerVolumeSpecName "kube-api-access-tptq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:15.409023 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.409009 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7da0f4c8-83ff-4a6f-8946-ef4988bba28a" (UID: "7da0f4c8-83ff-4a6f-8946-ef4988bba28a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:34:15.428207 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.428182 2586 scope.go:117] "RemoveContainer" containerID="ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503" Jul 09 17:34:15.428566 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:15.428538 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503\": container with ID starting with ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503 not found: ID does not exist" containerID="ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503" Jul 09 17:34:15.428661 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.428581 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503"} err="failed to get container status \"ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503\": rpc error: code = NotFound desc = could not find container \"ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503\": container with ID starting with ca0468b07dc5e48fa1c2d17b14bfad746aced218d6d8a9e51149b4834f3a6503 not found: ID does not exist" Jul 09 17:34:15.508409 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.508306 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:15.508409 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.508343 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tptq6\" (UniqueName: \"kubernetes.io/projected/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-kube-api-access-tptq6\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:15.508409 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.508359 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7da0f4c8-83ff-4a6f-8946-ef4988bba28a-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:15.729014 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.728979 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq"] Jul 09 17:34:15.732990 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:15.732960 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2e22ec17-75d6c994db-8z5vq"] Jul 09 17:34:16.672098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:16.672065 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7da0f4c8-83ff-4a6f-8946-ef4988bba28a" path="/var/lib/kubelet/pods/7da0f4c8-83ff-4a6f-8946-ef4988bba28a/volumes" Jul 09 17:34:20.453574 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:20.453536 2586 generic.go:356] "Generic (PLEG): container finished" podID="7311ac88-0260-437b-a074-6e53896ed8d3" containerID="19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54" exitCode=0 Jul 09 17:34:20.454177 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:20.453616 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" event={"ID":"7311ac88-0260-437b-a074-6e53896ed8d3","Type":"ContainerDied","Data":"19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54"} Jul 09 17:34:20.454265 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:20.454186 2586 scope.go:117] "RemoveContainer" containerID="19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54" Jul 09 17:34:20.455240 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:20.455214 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:20.455312 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:20.455299 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:20.455350 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:20.455332 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:20.678465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:20.678420 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq"] Jul 09 17:34:21.461116 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:21.461080 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" event={"ID":"7311ac88-0260-437b-a074-6e53896ed8d3","Type":"ContainerStarted","Data":"34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df"} Jul 09 17:34:21.461632 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:21.461198 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" podUID="7311ac88-0260-437b-a074-6e53896ed8d3" containerName="istio-proxy" containerID="cri-o://34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df" gracePeriod=30 Jul 09 17:34:25.360524 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:25.360484 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:26.684809 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.684775 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq"] Jul 09 17:34:26.685524 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.685488 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da0f4c8-83ff-4a6f-8946-ef4988bba28a" containerName="maas-api" Jul 09 17:34:26.685524 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.685514 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da0f4c8-83ff-4a6f-8946-ef4988bba28a" containerName="maas-api" Jul 09 17:34:26.685705 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.685634 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7da0f4c8-83ff-4a6f-8946-ef4988bba28a" containerName="maas-api" Jul 09 17:34:26.699166 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.699119 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.702187 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.702153 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq"] Jul 09 17:34:26.704408 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.704372 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-61ba4a70-openshift-default-dockercfg-rc5dx\"" Jul 09 17:34:26.742173 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.742143 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:26.830167 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830122 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-data\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.830383 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830252 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-credential-socket\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.830383 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830276 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebe4926f-1af0-4528-862f-6e25c826fca3-istiod-ca-cert\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.830383 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830330 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-podinfo\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.830629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830387 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-envoy\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.830629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830432 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-token\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.830629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830535 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-socket\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.830629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830562 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ww75\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-kube-api-access-5ww75\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.830823 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.830655 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-certs\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.931625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931584 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7311ac88-0260-437b-a074-6e53896ed8d3-istiod-ca-cert\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.931834 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931638 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-envoy\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.931834 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931674 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-credential-socket\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.931834 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931774 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-certs\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.931834 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931826 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-socket\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.932060 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931857 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46g29\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-kube-api-access-46g29\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.932060 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931880 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7311ac88-0260-437b-a074-6e53896ed8d3-istio-podinfo\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.932060 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931937 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-data\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.932060 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932000 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-istio-token\") pod \"7311ac88-0260-437b-a074-6e53896ed8d3\" (UID: \"7311ac88-0260-437b-a074-6e53896ed8d3\") " Jul 09 17:34:26.932060 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.931980 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:26.932060 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932039 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7311ac88-0260-437b-a074-6e53896ed8d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:34:26.932365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932056 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:26.932365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932165 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:26.932365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932214 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-credential-socket\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932253 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebe4926f-1af0-4528-862f-6e25c826fca3-istiod-ca-cert\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932302 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-podinfo\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932335 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-envoy\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932365 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-token\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932401 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-socket\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932429 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ww75\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-kube-api-access-5ww75\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932538 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-certs\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932580 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-credential-socket\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932637 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-data\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.932984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932749 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7311ac88-0260-437b-a074-6e53896ed8d3-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:26.932984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932766 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:26.932984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932780 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:26.932984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932799 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:26.932984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.932849 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-certs\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.933215 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.933014 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-data\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.933215 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.933029 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:26.933215 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.933139 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebe4926f-1af0-4528-862f-6e25c826fca3-istiod-ca-cert\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.933644 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.933394 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-socket\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.934556 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.934528 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:26.934698 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.934611 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:26.934698 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.934667 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-kube-api-access-46g29" (OuterVolumeSpecName: "kube-api-access-46g29") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "kube-api-access-46g29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:26.935088 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.935071 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7311ac88-0260-437b-a074-6e53896ed8d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7311ac88-0260-437b-a074-6e53896ed8d3" (UID: "7311ac88-0260-437b-a074-6e53896ed8d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:34:26.935643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.935618 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-envoy\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.935816 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.935800 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-podinfo\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.941764 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.941740 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-token\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:26.942337 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:26.942315 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ww75\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-kube-api-access-5ww75\") pod \"e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:27.016416 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.016303 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:27.034342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.034311 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46g29\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-kube-api-access-46g29\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:27.034342 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.034342 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7311ac88-0260-437b-a074-6e53896ed8d3-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:27.034534 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.034353 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:27.034534 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.034361 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7311ac88-0260-437b-a074-6e53896ed8d3-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:27.034534 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.034369 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7311ac88-0260-437b-a074-6e53896ed8d3-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:27.163996 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.163967 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq"] Jul 09 17:34:27.165738 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:34:27.165713 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe4926f_1af0_4528_862f_6e25c826fca3.slice/crio-cfefaede79a2cd66a723162324dd8e2ea3af46d09c0bc72e9253761f4fec699c WatchSource:0}: Error finding container cfefaede79a2cd66a723162324dd8e2ea3af46d09c0bc72e9253761f4fec699c: Status 404 returned error can't find the container with id cfefaede79a2cd66a723162324dd8e2ea3af46d09c0bc72e9253761f4fec699c Jul 09 17:34:27.167947 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.167919 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:27.168029 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.167985 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:27.168029 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.168013 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:34:27.495163 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.495129 2586 generic.go:356] "Generic (PLEG): container finished" podID="7311ac88-0260-437b-a074-6e53896ed8d3" containerID="34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df" exitCode=0 Jul 09 17:34:27.495374 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.495189 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" Jul 09 17:34:27.495374 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.495223 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" event={"ID":"7311ac88-0260-437b-a074-6e53896ed8d3","Type":"ContainerDied","Data":"34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df"} Jul 09 17:34:27.495374 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.495288 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq" event={"ID":"7311ac88-0260-437b-a074-6e53896ed8d3","Type":"ContainerDied","Data":"1c658c09bd343550e39a77e9f87417f5bc741e5fccff54474f58329cb19c96f5"} Jul 09 17:34:27.495374 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.495311 2586 scope.go:117] "RemoveContainer" containerID="34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df" Jul 09 17:34:27.497149 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.497118 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" event={"ID":"ebe4926f-1af0-4528-862f-6e25c826fca3","Type":"ContainerStarted","Data":"26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a"} Jul 09 17:34:27.497305 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.497175 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" event={"ID":"ebe4926f-1af0-4528-862f-6e25c826fca3","Type":"ContainerStarted","Data":"cfefaede79a2cd66a723162324dd8e2ea3af46d09c0bc72e9253761f4fec699c"} Jul 09 17:34:27.506022 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.506005 2586 scope.go:117] "RemoveContainer" containerID="19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54" Jul 09 17:34:27.515935 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.515915 2586 scope.go:117] "RemoveContainer" containerID="34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df" Jul 09 17:34:27.516239 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:27.516219 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df\": container with ID starting with 34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df not found: ID does not exist" containerID="34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df" Jul 09 17:34:27.516289 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.516250 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df"} err="failed to get container status \"34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df\": rpc error: code = NotFound desc = could not find container \"34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df\": container with ID starting with 34da4e4747d70093a44ccd79be024d5694a7a2ccb9b1d4bd24d2b590ec2828df not found: ID does not exist" Jul 09 17:34:27.516289 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.516268 2586 scope.go:117] "RemoveContainer" containerID="19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54" Jul 09 17:34:27.516571 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:27.516552 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54\": container with ID starting with 19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54 not found: ID does not exist" containerID="19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54" Jul 09 17:34:27.516628 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.516580 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54"} err="failed to get container status \"19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54\": rpc error: code = NotFound desc = could not find container \"19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54\": container with ID starting with 19c1108500ffc60be14d6207ed7846ccfae18f99dd14fcebca1a7c921aee0e54 not found: ID does not exist" Jul 09 17:34:27.522596 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.522535 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" podStartSLOduration=1.522516113 podStartE2EDuration="1.522516113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:34:27.52001019 +0000 UTC m=+1527.449455724" watchObservedRunningTime="2026-07-09 17:34:27.522516113 +0000 UTC m=+1527.451961603" Jul 09 17:34:27.534515 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.534470 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq"] Jul 09 17:34:27.544062 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.544035 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2e22ec17-openshift-default-7c4bcd9bf7-78txq"] Jul 09 17:34:27.935634 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.935509 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl"] Jul 09 17:34:27.936229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.936137 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7311ac88-0260-437b-a074-6e53896ed8d3" containerName="istio-proxy" Jul 09 17:34:27.936229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.936158 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7311ac88-0260-437b-a074-6e53896ed8d3" containerName="istio-proxy" Jul 09 17:34:27.936229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.936199 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7311ac88-0260-437b-a074-6e53896ed8d3" containerName="istio-proxy" Jul 09 17:34:27.936229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.936208 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7311ac88-0260-437b-a074-6e53896ed8d3" containerName="istio-proxy" Jul 09 17:34:27.936433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.936321 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7311ac88-0260-437b-a074-6e53896ed8d3" containerName="istio-proxy" Jul 09 17:34:27.936433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.936334 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7311ac88-0260-437b-a074-6e53896ed8d3" containerName="istio-proxy" Jul 09 17:34:27.941182 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.941157 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:27.941738 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.941706 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-openshift-service-ca\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:27.941861 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.941773 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx5kn\" (UniqueName: \"kubernetes.io/projected/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-kube-api-access-lx5kn\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:27.941959 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.941934 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-maas-api-tls\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:27.944141 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.944116 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-61ba4a70-serving-cert\"" Jul 09 17:34:27.950560 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:27.950529 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl"] Jul 09 17:34:28.016979 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.016942 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:28.021924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.021895 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:28.042951 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.042915 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-maas-api-tls\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:28.043172 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.042990 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-openshift-service-ca\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:28.043172 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.043026 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx5kn\" (UniqueName: \"kubernetes.io/projected/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-kube-api-access-lx5kn\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:28.043751 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.043727 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-openshift-service-ca\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:28.045404 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.045377 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-maas-api-tls\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:28.054629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.054600 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx5kn\" (UniqueName: \"kubernetes.io/projected/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-kube-api-access-lx5kn\") pod \"maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:28.254825 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.254786 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:28.396620 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.396593 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl"] Jul 09 17:34:28.398585 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:34:28.398560 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc586aa7b_8e16_4c8b_bdb4_f6b1b58135c0.slice/crio-bf22791e6419e4fc8c5c325ea46861605d62acb84ffc968acb6f26de6d046d18 WatchSource:0}: Error finding container bf22791e6419e4fc8c5c325ea46861605d62acb84ffc968acb6f26de6d046d18: Status 404 returned error can't find the container with id bf22791e6419e4fc8c5c325ea46861605d62acb84ffc968acb6f26de6d046d18 Jul 09 17:34:28.503644 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.503610 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" event={"ID":"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0","Type":"ContainerStarted","Data":"bf22791e6419e4fc8c5c325ea46861605d62acb84ffc968acb6f26de6d046d18"} Jul 09 17:34:28.504822 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.504791 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:28.505717 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.505660 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:28.672973 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:28.672938 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7311ac88-0260-437b-a074-6e53896ed8d3" path="/var/lib/kubelet/pods/7311ac88-0260-437b-a074-6e53896ed8d3/volumes" Jul 09 17:34:29.511376 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:29.511335 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" event={"ID":"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0","Type":"ContainerStarted","Data":"ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523"} Jul 09 17:34:29.511807 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:29.511548 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:29.529998 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:29.529941 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" podStartSLOduration=2.199690102 podStartE2EDuration="2.529924271s" totalImagesPullingTime="330.234169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:34:29.527740405 +0000 UTC m=+1529.457185917" watchObservedRunningTime="2026-07-09 17:34:29.529924271 +0000 UTC m=+1529.459369826" Jul 09 17:34:32.512985 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:32.512942 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq"] Jul 09 17:34:32.513466 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:32.513140 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" podUID="ebe4926f-1af0-4528-862f-6e25c826fca3" containerName="istio-proxy" containerID="cri-o://26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a" gracePeriod=30 Jul 09 17:34:32.682593 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:32.682556 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl"] Jul 09 17:34:32.683075 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:32.683040 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" podUID="c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" containerName="maas-api" containerID="cri-o://ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523" gracePeriod=30 Jul 09 17:34:32.971588 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:32.971562 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:33.006548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.005717 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-maas-api-tls\") pod \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " Jul 09 17:34:33.006548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.005791 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx5kn\" (UniqueName: \"kubernetes.io/projected/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-kube-api-access-lx5kn\") pod \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " Jul 09 17:34:33.006548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.005858 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-openshift-service-ca\") pod \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\" (UID: \"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0\") " Jul 09 17:34:33.006548 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.006480 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" (UID: "c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:34:33.008489 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.008460 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" (UID: "c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:34:33.009270 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.009238 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-kube-api-access-lx5kn" (OuterVolumeSpecName: "kube-api-access-lx5kn") pod "c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" (UID: "c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0"). InnerVolumeSpecName "kube-api-access-lx5kn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:33.107413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.107317 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:33.107413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.107351 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx5kn\" (UniqueName: \"kubernetes.io/projected/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-kube-api-access-lx5kn\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:33.107413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.107360 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:33.534859 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.534825 2586 generic.go:356] "Generic (PLEG): container finished" podID="c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" containerID="ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523" exitCode=0 Jul 09 17:34:33.535286 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.534865 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" Jul 09 17:34:33.535286 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.534874 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" event={"ID":"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0","Type":"ContainerDied","Data":"ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523"} Jul 09 17:34:33.535286 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.534920 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl" event={"ID":"c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0","Type":"ContainerDied","Data":"bf22791e6419e4fc8c5c325ea46861605d62acb84ffc968acb6f26de6d046d18"} Jul 09 17:34:33.535286 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.534943 2586 scope.go:117] "RemoveContainer" containerID="ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523" Jul 09 17:34:33.544610 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.544592 2586 scope.go:117] "RemoveContainer" containerID="ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523" Jul 09 17:34:33.544893 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:33.544870 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523\": container with ID starting with ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523 not found: ID does not exist" containerID="ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523" Jul 09 17:34:33.544973 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.544901 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523"} err="failed to get container status \"ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523\": rpc error: code = NotFound desc = could not find container \"ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523\": container with ID starting with ef24c0773cd1875ba912fe5c53f06e45e226ed7289e4b03feba0a54eba1c3523 not found: ID does not exist" Jul 09 17:34:33.557476 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.557404 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl"] Jul 09 17:34:33.560167 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:33.560138 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61ba4a70-84f8445564-8wdbl"] Jul 09 17:34:34.675647 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:34.673962 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" path="/var/lib/kubelet/pods/c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0/volumes" Jul 09 17:34:38.788449 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.788415 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:38.863137 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863038 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ww75\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-kube-api-access-5ww75\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863137 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863128 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-podinfo\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863378 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863155 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-credential-socket\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863378 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863181 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-socket\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863378 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863318 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebe4926f-1af0-4528-862f-6e25c826fca3-istiod-ca-cert\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863566 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863389 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-token\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863566 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863474 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-envoy\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863566 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863509 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-certs\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863566 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863510 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:38.863787 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863599 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-data\") pod \"ebe4926f-1af0-4528-862f-6e25c826fca3\" (UID: \"ebe4926f-1af0-4528-862f-6e25c826fca3\") " Jul 09 17:34:38.863787 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863771 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:38.863897 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863789 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:38.863897 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.863812 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe4926f-1af0-4528-862f-6e25c826fca3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:34:38.864117 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.864093 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:38.864269 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.864121 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:38.864269 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.864137 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:38.864269 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.864152 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ebe4926f-1af0-4528-862f-6e25c826fca3-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:38.864768 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.864741 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-data" (OuterVolumeSpecName: "istio-data") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:38.865375 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.865354 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:34:38.865559 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.865469 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-kube-api-access-5ww75" (OuterVolumeSpecName: "kube-api-access-5ww75") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "kube-api-access-5ww75". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:38.865609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.865568 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-token" (OuterVolumeSpecName: "istio-token") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:34:38.865774 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.865747 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ebe4926f-1af0-4528-862f-6e25c826fca3" (UID: "ebe4926f-1af0-4528-862f-6e25c826fca3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:34:38.965734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.965683 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:38.965734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.965727 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:38.965734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.965736 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:38.965734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.965743 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ebe4926f-1af0-4528-862f-6e25c826fca3-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:38.965734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:38.965752 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ww75\" (UniqueName: \"kubernetes.io/projected/ebe4926f-1af0-4528-862f-6e25c826fca3-kube-api-access-5ww75\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:34:39.568923 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.568888 2586 generic.go:356] "Generic (PLEG): container finished" podID="ebe4926f-1af0-4528-862f-6e25c826fca3" containerID="26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a" exitCode=0 Jul 09 17:34:39.569098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.568934 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" Jul 09 17:34:39.569098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.568949 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" event={"ID":"ebe4926f-1af0-4528-862f-6e25c826fca3","Type":"ContainerDied","Data":"26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a"} Jul 09 17:34:39.569098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.568994 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq" event={"ID":"ebe4926f-1af0-4528-862f-6e25c826fca3","Type":"ContainerDied","Data":"cfefaede79a2cd66a723162324dd8e2ea3af46d09c0bc72e9253761f4fec699c"} Jul 09 17:34:39.569098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.569014 2586 scope.go:117] "RemoveContainer" containerID="26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a" Jul 09 17:34:39.578354 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.578336 2586 scope.go:117] "RemoveContainer" containerID="26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a" Jul 09 17:34:39.578625 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:34:39.578606 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a\": container with ID starting with 26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a not found: ID does not exist" containerID="26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a" Jul 09 17:34:39.578671 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.578632 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a"} err="failed to get container status \"26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a\": rpc error: code = NotFound desc = could not find container \"26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a\": container with ID starting with 26fca2a60e6cc3a9608829259237ee73308e7c18176e325f16a0ddc2fce3f08a not found: ID does not exist" Jul 09 17:34:39.591262 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.591236 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq"] Jul 09 17:34:39.600985 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:39.600952 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-61ba4a70-openshift-default-7498d85ff5-9l5mq"] Jul 09 17:34:40.671644 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:34:40.671612 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebe4926f-1af0-4528-862f-6e25c826fca3" path="/var/lib/kubelet/pods/ebe4926f-1af0-4528-862f-6e25c826fca3/volumes" Jul 09 17:36:12.791136 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.790938 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g"] Jul 09 17:36:12.792764 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.792732 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe4926f-1af0-4528-862f-6e25c826fca3" containerName="istio-proxy" Jul 09 17:36:12.792764 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.792762 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe4926f-1af0-4528-862f-6e25c826fca3" containerName="istio-proxy" Jul 09 17:36:12.792946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.792791 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" containerName="maas-api" Jul 09 17:36:12.792946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.792800 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" containerName="maas-api" Jul 09 17:36:12.793046 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.792963 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="c586aa7b-8e16-4c8b-bdb4-f6b1b58135c0" containerName="maas-api" Jul 09 17:36:12.793046 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.792986 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebe4926f-1af0-4528-862f-6e25c826fca3" containerName="istio-proxy" Jul 09 17:36:12.799404 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.799379 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.802472 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.802450 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8196e115-openshift-default-dockercfg-6tkbx\"" Jul 09 17:36:12.816915 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.816893 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g"] Jul 09 17:36:12.878778 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.878736 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-podinfo\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.878984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.878801 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-envoy\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.878984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.878882 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-token\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.879105 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.878980 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-credential-socket\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.879105 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.879012 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-socket\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.879105 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.879038 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjsz2\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-kube-api-access-jjsz2\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.879105 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.879091 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-certs\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.879323 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.879115 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-data\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.879323 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.879150 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istiod-ca-cert\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980283 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980250 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-credential-socket\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980283 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980283 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-socket\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980537 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980303 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjsz2\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-kube-api-access-jjsz2\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980537 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980352 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-certs\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980537 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980376 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-data\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980755 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980535 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istiod-ca-cert\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980755 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980626 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-podinfo\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980755 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980697 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-envoy\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980755 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980696 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-socket\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980965 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980756 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-data\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980965 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980779 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-token\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980965 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980802 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-certs\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.980965 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.980817 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-credential-socket\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.981312 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.981286 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istiod-ca-cert\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.982932 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.982909 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-envoy\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.983014 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.982934 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-podinfo\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.988851 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.988823 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjsz2\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-kube-api-access-jjsz2\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:12.989381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:12.989365 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-token\") pod \"e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:13.110115 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:13.110000 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:13.248261 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:13.248236 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g"] Jul 09 17:36:13.251084 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:36:13.251056 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccccdae4_6e3a_46ca_9ed2_4ec4019d0147.slice/crio-2810ee5f2f8a4dc250ee0d1f919e13647f96a605b490284b490c955081d61397 WatchSource:0}: Error finding container 2810ee5f2f8a4dc250ee0d1f919e13647f96a605b490284b490c955081d61397: Status 404 returned error can't find the container with id 2810ee5f2f8a4dc250ee0d1f919e13647f96a605b490284b490c955081d61397 Jul 09 17:36:13.253225 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:13.253194 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:13.253278 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:13.253265 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:13.253314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:13.253293 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:14.039919 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:14.039880 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" event={"ID":"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147","Type":"ContainerStarted","Data":"a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518"} Jul 09 17:36:14.040311 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:14.039936 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" event={"ID":"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147","Type":"ContainerStarted","Data":"2810ee5f2f8a4dc250ee0d1f919e13647f96a605b490284b490c955081d61397"} Jul 09 17:36:14.060632 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:14.060576 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" podStartSLOduration=2.060544316 podStartE2EDuration="2.060544316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:36:14.058805701 +0000 UTC m=+1633.988251238" watchObservedRunningTime="2026-07-09 17:36:14.060544316 +0000 UTC m=+1633.989989828" Jul 09 17:36:14.110630 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:14.110589 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:14.115323 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:14.115299 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:15.045030 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:15.044995 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:15.045954 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:15.045936 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:16.190506 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.190466 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8"] Jul 09 17:36:16.193159 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.193126 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.196145 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.196122 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8196e115-serving-cert\"" Jul 09 17:36:16.203135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.203111 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8"] Jul 09 17:36:16.314918 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.314881 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll69l\" (UniqueName: \"kubernetes.io/projected/87236388-81d6-41f0-a6e7-df295df85a1b-kube-api-access-ll69l\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.315098 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.314932 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.315141 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.315091 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87236388-81d6-41f0-a6e7-df295df85a1b-openshift-service-ca\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.416165 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.416112 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87236388-81d6-41f0-a6e7-df295df85a1b-openshift-service-ca\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.416348 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.416186 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll69l\" (UniqueName: \"kubernetes.io/projected/87236388-81d6-41f0-a6e7-df295df85a1b-kube-api-access-ll69l\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.416348 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.416215 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.416348 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:16.416337 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-serving-cert: secret "maas-api-e2e-mt-8196e115-serving-cert" not found Jul 09 17:36:16.416493 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:16.416394 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls podName:87236388-81d6-41f0-a6e7-df295df85a1b nodeName:}" failed. No retries permitted until 2026-07-09 17:36:16.916376537 +0000 UTC m=+1636.845822026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls") pod "maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" (UID: "87236388-81d6-41f0-a6e7-df295df85a1b") : secret "maas-api-e2e-mt-8196e115-serving-cert" not found Jul 09 17:36:16.416818 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.416793 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87236388-81d6-41f0-a6e7-df295df85a1b-openshift-service-ca\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.425596 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.425570 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll69l\" (UniqueName: \"kubernetes.io/projected/87236388-81d6-41f0-a6e7-df295df85a1b-kube-api-access-ll69l\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.920989 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:16.920949 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:16.921206 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:16.921130 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-serving-cert: secret "maas-api-e2e-mt-8196e115-serving-cert" not found Jul 09 17:36:16.921273 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:16.921224 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls podName:87236388-81d6-41f0-a6e7-df295df85a1b nodeName:}" failed. No retries permitted until 2026-07-09 17:36:17.921202925 +0000 UTC m=+1637.850648421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls") pod "maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" (UID: "87236388-81d6-41f0-a6e7-df295df85a1b") : secret "maas-api-e2e-mt-8196e115-serving-cert" not found Jul 09 17:36:17.932304 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:17.932264 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:17.934927 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:17.934899 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls\") pod \"maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:18.005805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:18.005769 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:18.143721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:18.143690 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8"] Jul 09 17:36:18.145918 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:36:18.145886 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87236388_81d6_41f0_a6e7_df295df85a1b.slice/crio-f60af0e938e6c57ed0f4033ac973c8cf4713ffa15d6f952c787f0ab432053857 WatchSource:0}: Error finding container f60af0e938e6c57ed0f4033ac973c8cf4713ffa15d6f952c787f0ab432053857: Status 404 returned error can't find the container with id f60af0e938e6c57ed0f4033ac973c8cf4713ffa15d6f952c787f0ab432053857 Jul 09 17:36:19.068163 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:19.068070 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" event={"ID":"87236388-81d6-41f0-a6e7-df295df85a1b","Type":"ContainerStarted","Data":"9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6"} Jul 09 17:36:19.068163 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:19.068127 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" event={"ID":"87236388-81d6-41f0-a6e7-df295df85a1b","Type":"ContainerStarted","Data":"f60af0e938e6c57ed0f4033ac973c8cf4713ffa15d6f952c787f0ab432053857"} Jul 09 17:36:19.068163 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:19.068164 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:19.085319 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:19.085266 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" podStartSLOduration=2.466086921 podStartE2EDuration="3.08524827s" totalImagesPullingTime="619.161349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:36:19.083816396 +0000 UTC m=+1639.013261909" watchObservedRunningTime="2026-07-09 17:36:19.08524827 +0000 UTC m=+1639.014693782" Jul 09 17:36:25.083762 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:25.083722 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:28.116941 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.116894 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8"] Jul 09 17:36:28.118601 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.117936 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" podUID="87236388-81d6-41f0-a6e7-df295df85a1b" containerName="maas-api" containerID="cri-o://9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6" gracePeriod=30 Jul 09 17:36:28.398480 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.398454 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:28.443364 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.443324 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll69l\" (UniqueName: \"kubernetes.io/projected/87236388-81d6-41f0-a6e7-df295df85a1b-kube-api-access-ll69l\") pod \"87236388-81d6-41f0-a6e7-df295df85a1b\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " Jul 09 17:36:28.443582 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.443489 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87236388-81d6-41f0-a6e7-df295df85a1b-openshift-service-ca\") pod \"87236388-81d6-41f0-a6e7-df295df85a1b\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " Jul 09 17:36:28.443656 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.443645 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls\") pod \"87236388-81d6-41f0-a6e7-df295df85a1b\" (UID: \"87236388-81d6-41f0-a6e7-df295df85a1b\") " Jul 09 17:36:28.443887 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.443853 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87236388-81d6-41f0-a6e7-df295df85a1b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "87236388-81d6-41f0-a6e7-df295df85a1b" (UID: "87236388-81d6-41f0-a6e7-df295df85a1b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:36:28.444044 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.444021 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/87236388-81d6-41f0-a6e7-df295df85a1b-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:28.445655 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.445627 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87236388-81d6-41f0-a6e7-df295df85a1b-kube-api-access-ll69l" (OuterVolumeSpecName: "kube-api-access-ll69l") pod "87236388-81d6-41f0-a6e7-df295df85a1b" (UID: "87236388-81d6-41f0-a6e7-df295df85a1b"). InnerVolumeSpecName "kube-api-access-ll69l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:36:28.445833 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.445815 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "87236388-81d6-41f0-a6e7-df295df85a1b" (UID: "87236388-81d6-41f0-a6e7-df295df85a1b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:36:28.545132 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.545081 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/87236388-81d6-41f0-a6e7-df295df85a1b-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:28.545132 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:28.545126 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll69l\" (UniqueName: \"kubernetes.io/projected/87236388-81d6-41f0-a6e7-df295df85a1b-kube-api-access-ll69l\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:29.135370 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.135331 2586 generic.go:356] "Generic (PLEG): container finished" podID="87236388-81d6-41f0-a6e7-df295df85a1b" containerID="9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6" exitCode=0 Jul 09 17:36:29.135829 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.135401 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" Jul 09 17:36:29.135829 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.135398 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" event={"ID":"87236388-81d6-41f0-a6e7-df295df85a1b","Type":"ContainerDied","Data":"9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6"} Jul 09 17:36:29.135829 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.135503 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8" event={"ID":"87236388-81d6-41f0-a6e7-df295df85a1b","Type":"ContainerDied","Data":"f60af0e938e6c57ed0f4033ac973c8cf4713ffa15d6f952c787f0ab432053857"} Jul 09 17:36:29.135829 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.135532 2586 scope.go:117] "RemoveContainer" containerID="9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6" Jul 09 17:36:29.146722 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.146695 2586 scope.go:117] "RemoveContainer" containerID="9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6" Jul 09 17:36:29.147013 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:29.146990 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6\": container with ID starting with 9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6 not found: ID does not exist" containerID="9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6" Jul 09 17:36:29.147104 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.147027 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6"} err="failed to get container status \"9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6\": rpc error: code = NotFound desc = could not find container \"9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6\": container with ID starting with 9f97e1b9fd900991a483d2a78d46cd3d2173edd171815325ec14dc1bdc3c93d6 not found: ID does not exist" Jul 09 17:36:29.155564 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.155523 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8"] Jul 09 17:36:29.158946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:29.158918 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8196e115-8548f4fb98-f8zm8"] Jul 09 17:36:30.047741 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.047462 2586 patch_prober.go:28] interesting pod/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:48690->10.134.0.41:15021: read: connection reset by peer" start-of-body= Jul 09 17:36:30.047741 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.047548 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:48690->10.134.0.41:15021: read: connection reset by peer" Jul 09 17:36:30.142619 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.142583 2586 generic.go:356] "Generic (PLEG): container finished" podID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerID="a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518" exitCode=0 Jul 09 17:36:30.143029 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.142657 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" event={"ID":"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147","Type":"ContainerDied","Data":"a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518"} Jul 09 17:36:30.143216 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.143198 2586 scope.go:117] "RemoveContainer" containerID="a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518" Jul 09 17:36:30.144319 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.144290 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:30.144416 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.144360 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:30.144416 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.144394 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:30.672324 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:30.672287 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87236388-81d6-41f0-a6e7-df295df85a1b" path="/var/lib/kubelet/pods/87236388-81d6-41f0-a6e7-df295df85a1b/volumes" Jul 09 17:36:31.150477 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:31.150421 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" event={"ID":"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147","Type":"ContainerStarted","Data":"229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62"} Jul 09 17:36:32.111123 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:32.111090 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:32.111300 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:32.111246 2586 patch_prober.go:28] interesting pod/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 09 17:36:32.111300 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:32.111290 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 09 17:36:33.110782 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:33.110741 2586 patch_prober.go:28] interesting pod/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 09 17:36:33.111201 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:33.110799 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 09 17:36:34.094465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:34.094023 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g"] Jul 09 17:36:34.094465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:34.094303 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" containerID="cri-o://229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62" gracePeriod=30 Jul 09 17:36:40.373891 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.373859 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:40.474564 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474528 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-envoy\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.474564 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474565 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-data\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.474796 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474602 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-socket\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.474796 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474724 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-credential-socket\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.474796 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474751 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istiod-ca-cert\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.474796 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474786 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-podinfo\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.475017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474850 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjsz2\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-kube-api-access-jjsz2\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.475017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474883 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-certs\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.475017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474901 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:36:40.475017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474933 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-token\") pod \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\" (UID: \"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147\") " Jul 09 17:36:40.475017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.474992 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:36:40.475344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.475140 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:36:40.475344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.475163 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:36:40.475465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.475355 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:40.475465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.475373 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:40.475465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.475387 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:40.475465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.475399 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:40.475623 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.475537 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-data" (OuterVolumeSpecName: "istio-data") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:36:40.477020 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.476986 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-kube-api-access-jjsz2" (OuterVolumeSpecName: "kube-api-access-jjsz2") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "kube-api-access-jjsz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:36:40.477153 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.477083 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:36:40.477253 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.477237 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:36:40.477345 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.477329 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-token" (OuterVolumeSpecName: "istio-token") pod "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" (UID: "ccccdae4-6e3a-46ca-9ed2-4ec4019d0147"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:36:40.576902 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.576858 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjsz2\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-kube-api-access-jjsz2\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:40.576902 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.576891 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:40.576902 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.576901 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:40.576902 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.576911 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:40.577187 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:40.576918 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:36:41.207157 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.207117 2586 generic.go:356] "Generic (PLEG): container finished" podID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerID="229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62" exitCode=0 Jul 09 17:36:41.207353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.207180 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" Jul 09 17:36:41.207353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.207199 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" event={"ID":"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147","Type":"ContainerDied","Data":"229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62"} Jul 09 17:36:41.207353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.207263 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g" event={"ID":"ccccdae4-6e3a-46ca-9ed2-4ec4019d0147","Type":"ContainerDied","Data":"2810ee5f2f8a4dc250ee0d1f919e13647f96a605b490284b490c955081d61397"} Jul 09 17:36:41.207353 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.207297 2586 scope.go:117] "RemoveContainer" containerID="229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62" Jul 09 17:36:41.216425 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.216402 2586 scope.go:117] "RemoveContainer" containerID="a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518" Jul 09 17:36:41.226401 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.226228 2586 scope.go:117] "RemoveContainer" containerID="229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62" Jul 09 17:36:41.226642 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:41.226620 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62\": container with ID starting with 229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62 not found: ID does not exist" containerID="229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62" Jul 09 17:36:41.226703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.226652 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62"} err="failed to get container status \"229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62\": rpc error: code = NotFound desc = could not find container \"229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62\": container with ID starting with 229f9ef4d6dc40217c274baf0f0b941a84ca06aa8bc064064d72e06799fa9d62 not found: ID does not exist" Jul 09 17:36:41.226703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.226672 2586 scope.go:117] "RemoveContainer" containerID="a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518" Jul 09 17:36:41.226959 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:41.226939 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518\": container with ID starting with a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518 not found: ID does not exist" containerID="a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518" Jul 09 17:36:41.227023 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.226969 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518"} err="failed to get container status \"a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518\": rpc error: code = NotFound desc = could not find container \"a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518\": container with ID starting with a05bd6c4c855de22c34ddf1eaa7aed88791bad2e8fc08ddf575a9727484e1518 not found: ID does not exist" Jul 09 17:36:41.227573 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.227554 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g"] Jul 09 17:36:41.234767 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:41.234733 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8196e115-openshift-default-8455dc76c9-pr75g"] Jul 09 17:36:42.671999 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:42.671966 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" path="/var/lib/kubelet/pods/ccccdae4-6e3a-46ca-9ed2-4ec4019d0147/volumes" Jul 09 17:36:53.629056 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629013 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk"] Jul 09 17:36:53.629721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629698 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" Jul 09 17:36:53.629801 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629724 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" Jul 09 17:36:53.629801 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629743 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" Jul 09 17:36:53.629801 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629752 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" Jul 09 17:36:53.629801 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629776 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87236388-81d6-41f0-a6e7-df295df85a1b" containerName="maas-api" Jul 09 17:36:53.629801 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629786 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="87236388-81d6-41f0-a6e7-df295df85a1b" containerName="maas-api" Jul 09 17:36:53.630045 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629897 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="87236388-81d6-41f0-a6e7-df295df85a1b" containerName="maas-api" Jul 09 17:36:53.630045 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629910 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" Jul 09 17:36:53.630045 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.629920 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccccdae4-6e3a-46ca-9ed2-4ec4019d0147" containerName="istio-proxy" Jul 09 17:36:53.633453 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.633414 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.638150 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.638124 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7565f4e1-openshift-default-dockercfg-9xwzr\"" Jul 09 17:36:53.649934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.649901 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk"] Jul 09 17:36:53.805616 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.805582 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-podinfo\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.805616 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.805622 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istiod-ca-cert\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.805862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.805672 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-data\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.805862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.805697 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-certs\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.805933 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.805912 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-socket\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.805993 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.805970 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-token\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.806059 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.806022 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-credential-socket\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.806107 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.806064 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-envoy\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.806151 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.806137 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd6xj\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-kube-api-access-nd6xj\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.906993 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-socket\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907040 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-token\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907085 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907064 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-credential-socket\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907189 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-envoy\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907230 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd6xj\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-kube-api-access-nd6xj\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907279 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-podinfo\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907315 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istiod-ca-cert\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907592 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907393 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-data\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907592 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907462 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-certs\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907592 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907472 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-socket\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907592 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907517 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-credential-socket\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907816 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907785 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-data\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.907921 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.907894 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-certs\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.908057 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.908033 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istiod-ca-cert\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.909431 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.909415 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-envoy\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.909843 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.909819 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-podinfo\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.915309 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.915278 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-token\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.915481 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.915463 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd6xj\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-kube-api-access-nd6xj\") pod \"e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:53.944494 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:53.944431 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:54.087564 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:54.087537 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk"] Jul 09 17:36:54.089643 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:36:54.089615 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb89f6ef0_3092_4aa3_a3f7_0c89c85b6149.slice/crio-979bba43804168ed1c0ecfe82cc991747cc55318abb9fc119c8eb8dfb2f7dad2 WatchSource:0}: Error finding container 979bba43804168ed1c0ecfe82cc991747cc55318abb9fc119c8eb8dfb2f7dad2: Status 404 returned error can't find the container with id 979bba43804168ed1c0ecfe82cc991747cc55318abb9fc119c8eb8dfb2f7dad2 Jul 09 17:36:54.092619 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:54.092581 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:54.092698 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:54.092664 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:54.092738 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:54.092708 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:36:54.277125 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:54.277085 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" event={"ID":"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149","Type":"ContainerStarted","Data":"e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1"} Jul 09 17:36:54.277319 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:54.277150 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" event={"ID":"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149","Type":"ContainerStarted","Data":"979bba43804168ed1c0ecfe82cc991747cc55318abb9fc119c8eb8dfb2f7dad2"} Jul 09 17:36:54.302838 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:54.302772 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" podStartSLOduration=1.302750222 podStartE2EDuration="1.302750222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:36:54.299707091 +0000 UTC m=+1674.229152660" watchObservedRunningTime="2026-07-09 17:36:54.302750222 +0000 UTC m=+1674.232195736" Jul 09 17:36:54.945570 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:54.945533 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:55.950033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:55.950001 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:56.278696 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.278662 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm"] Jul 09 17:36:56.281383 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.281363 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.284623 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.284602 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7565f4e1-serving-cert\"" Jul 09 17:36:56.291322 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.290951 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:56.291787 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.291757 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm"] Jul 09 17:36:56.292018 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.291992 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:36:56.331507 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.331476 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.331771 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.331743 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg5gb\" (UniqueName: \"kubernetes.io/projected/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-kube-api-access-vg5gb\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.331905 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.331880 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-openshift-service-ca\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.433403 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.433354 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.433802 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.433411 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg5gb\" (UniqueName: \"kubernetes.io/projected/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-kube-api-access-vg5gb\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.433986 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:56.433959 2586 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-serving-cert: secret "maas-api-e2e-mt-7565f4e1-serving-cert" not found Jul 09 17:36:56.434127 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:36:56.434106 2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls podName:32a6d4ba-dd47-4948-8fd2-2bc3e59c966f nodeName:}" failed. No retries permitted until 2026-07-09 17:36:56.934056912 +0000 UTC m=+1676.863502405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls") pod "maas-api-e2e-mt-7565f4e1-744777c789-9sshm" (UID: "32a6d4ba-dd47-4948-8fd2-2bc3e59c966f") : secret "maas-api-e2e-mt-7565f4e1-serving-cert" not found Jul 09 17:36:56.434347 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.434264 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-openshift-service-ca\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.435193 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.435059 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-openshift-service-ca\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.442746 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.442713 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg5gb\" (UniqueName: \"kubernetes.io/projected/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-kube-api-access-vg5gb\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.939736 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.939692 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:56.942263 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:56.942227 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls\") pod \"maas-api-e2e-mt-7565f4e1-744777c789-9sshm\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:57.195219 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:57.195122 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:57.332856 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:57.332830 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm"] Jul 09 17:36:57.335130 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:36:57.335098 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a6d4ba_dd47_4948_8fd2_2bc3e59c966f.slice/crio-bdb070dd0c5491043466fef6639a02e5f8cdbd177c4bd9449a0001d00f105b56 WatchSource:0}: Error finding container bdb070dd0c5491043466fef6639a02e5f8cdbd177c4bd9449a0001d00f105b56: Status 404 returned error can't find the container with id bdb070dd0c5491043466fef6639a02e5f8cdbd177c4bd9449a0001d00f105b56 Jul 09 17:36:58.302250 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:58.302213 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" event={"ID":"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f","Type":"ContainerStarted","Data":"ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d"} Jul 09 17:36:58.302687 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:58.302281 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" event={"ID":"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f","Type":"ContainerStarted","Data":"bdb070dd0c5491043466fef6639a02e5f8cdbd177c4bd9449a0001d00f105b56"} Jul 09 17:36:58.302687 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:58.302460 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:36:58.323806 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:36:58.323756 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" podStartSLOduration=1.502091521 podStartE2EDuration="2.32373931s" totalImagesPullingTime="821.647789ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:36:58.322224522 +0000 UTC m=+1678.251670069" watchObservedRunningTime="2026-07-09 17:36:58.32373931 +0000 UTC m=+1678.253184875" Jul 09 17:37:04.312363 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:04.312327 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:37:12.332635 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:12.332595 2586 patch_prober.go:28] interesting pod/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 17:37:12.334514 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:12.332688 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 17:37:13.389892 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:13.389859 2586 generic.go:356] "Generic (PLEG): container finished" podID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerID="e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1" exitCode=0 Jul 09 17:37:13.389892 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:13.389896 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" event={"ID":"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149","Type":"ContainerDied","Data":"e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1"} Jul 09 17:37:13.390318 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:13.390291 2586 scope.go:117] "RemoveContainer" containerID="e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1" Jul 09 17:37:13.391347 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:13.391317 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:37:13.391490 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:13.391406 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:37:13.391490 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:13.391481 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:37:14.400885 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:14.400834 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" event={"ID":"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149","Type":"ContainerStarted","Data":"2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1"} Jul 09 17:37:14.945653 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:14.945614 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:37:14.945872 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:14.945811 2586 patch_prober.go:28] interesting pod/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 17:37:14.945872 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:14.945852 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 17:37:15.945286 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:15.945250 2586 patch_prober.go:28] interesting pod/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 17:37:15.945695 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:15.945309 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 17:37:16.945653 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:16.945613 2586 patch_prober.go:28] interesting pod/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 17:37:16.946120 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:16.945686 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 17:37:17.945721 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:17.945685 2586 patch_prober.go:28] interesting pod/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 09 17:37:17.946128 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:17.945742 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 09 17:37:18.949138 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:18.949103 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:37:18.949602 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:18.949395 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:37:18.950218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:18.950200 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:37:34.771457 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:34.771401 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm"] Jul 09 17:37:34.772096 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:34.771679 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" podUID="32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" containerName="maas-api" containerID="cri-o://ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d" gracePeriod=30 Jul 09 17:37:35.021806 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.021741 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:37:35.118424 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.118389 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls\") pod \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " Jul 09 17:37:35.118665 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.118511 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-openshift-service-ca\") pod \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " Jul 09 17:37:35.118665 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.118602 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg5gb\" (UniqueName: \"kubernetes.io/projected/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-kube-api-access-vg5gb\") pod \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\" (UID: \"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f\") " Jul 09 17:37:35.118915 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.118889 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" (UID: "32a6d4ba-dd47-4948-8fd2-2bc3e59c966f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:37:35.120592 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.120570 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" (UID: "32a6d4ba-dd47-4948-8fd2-2bc3e59c966f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:37:35.120676 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.120577 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-kube-api-access-vg5gb" (OuterVolumeSpecName: "kube-api-access-vg5gb") pod "32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" (UID: "32a6d4ba-dd47-4948-8fd2-2bc3e59c966f"). InnerVolumeSpecName "kube-api-access-vg5gb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:37:35.220072 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.220037 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg5gb\" (UniqueName: \"kubernetes.io/projected/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-kube-api-access-vg5gb\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:37:35.220072 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.220067 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:37:35.220072 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.220077 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:37:35.511894 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.511860 2586 generic.go:356] "Generic (PLEG): container finished" podID="32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" containerID="ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d" exitCode=0 Jul 09 17:37:35.512077 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.511911 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" Jul 09 17:37:35.512077 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.511949 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" event={"ID":"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f","Type":"ContainerDied","Data":"ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d"} Jul 09 17:37:35.512077 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.512005 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm" event={"ID":"32a6d4ba-dd47-4948-8fd2-2bc3e59c966f","Type":"ContainerDied","Data":"bdb070dd0c5491043466fef6639a02e5f8cdbd177c4bd9449a0001d00f105b56"} Jul 09 17:37:35.512077 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.512029 2586 scope.go:117] "RemoveContainer" containerID="ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d" Jul 09 17:37:35.521181 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.521164 2586 scope.go:117] "RemoveContainer" containerID="ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d" Jul 09 17:37:35.521483 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:37:35.521461 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d\": container with ID starting with ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d not found: ID does not exist" containerID="ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d" Jul 09 17:37:35.521529 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.521491 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d"} err="failed to get container status \"ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d\": rpc error: code = NotFound desc = could not find container \"ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d\": container with ID starting with ce4c947c007a0bcccba237f18496e2bcd37b9f59a0fca96b13ddb3372cea232d not found: ID does not exist" Jul 09 17:37:35.534146 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.534116 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm"] Jul 09 17:37:35.537809 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:35.537774 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7565f4e1-744777c789-9sshm"] Jul 09 17:37:36.671793 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:37:36.671755 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" path="/var/lib/kubelet/pods/32a6d4ba-dd47-4948-8fd2-2bc3e59c966f/volumes" Jul 09 17:38:34.008541 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:34.008503 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk"] Jul 09 17:38:34.009092 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:34.008714 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" containerID="cri-o://2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1" gracePeriod=30 Jul 09 17:38:39.274088 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.274023 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:38:39.446398 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446362 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-credential-socket\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446422 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-data\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446465 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-certs\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446488 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-socket\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446514 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-envoy\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446599 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-podinfo\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446906 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446629 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd6xj\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-kube-api-access-nd6xj\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446906 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446669 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-token\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446906 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446698 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istiod-ca-cert\") pod \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\" (UID: \"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149\") " Jul 09 17:38:39.446906 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446739 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:39.446906 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446766 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:39.446906 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.446749 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:39.447413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.447064 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.447413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.447069 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:38:39.447413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.447085 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.447413 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.447099 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.447603 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.447583 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-data" (OuterVolumeSpecName: "istio-data") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:39.448891 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.448867 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-kube-api-access-nd6xj" (OuterVolumeSpecName: "kube-api-access-nd6xj") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "kube-api-access-nd6xj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:38:39.448996 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.448894 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:39.449200 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.449175 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:38:39.449299 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.449197 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-token" (OuterVolumeSpecName: "istio-token") pod "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" (UID: "b89f6ef0-3092-4aa3-a3f7-0c89c85b6149"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:38:39.547689 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.547589 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.547689 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.547631 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd6xj\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-kube-api-access-nd6xj\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.547689 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.547643 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.547689 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.547652 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.547689 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.547661 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.547689 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.547670 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:38:39.847246 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.847156 2586 generic.go:356] "Generic (PLEG): container finished" podID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerID="2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1" exitCode=0 Jul 09 17:38:39.847246 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.847212 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" Jul 09 17:38:39.847246 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.847225 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" event={"ID":"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149","Type":"ContainerDied","Data":"2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1"} Jul 09 17:38:39.847527 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.847285 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk" event={"ID":"b89f6ef0-3092-4aa3-a3f7-0c89c85b6149","Type":"ContainerDied","Data":"979bba43804168ed1c0ecfe82cc991747cc55318abb9fc119c8eb8dfb2f7dad2"} Jul 09 17:38:39.847527 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.847313 2586 scope.go:117] "RemoveContainer" containerID="2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1" Jul 09 17:38:39.857008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.856990 2586 scope.go:117] "RemoveContainer" containerID="e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1" Jul 09 17:38:39.867010 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.866994 2586 scope.go:117] "RemoveContainer" containerID="2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1" Jul 09 17:38:39.867378 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:38:39.867360 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1\": container with ID starting with 2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1 not found: ID does not exist" containerID="2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1" Jul 09 17:38:39.867456 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.867387 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1"} err="failed to get container status \"2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1\": rpc error: code = NotFound desc = could not find container \"2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1\": container with ID starting with 2b775c893ef4318b5860cfc5f1c56ca641ad23abd9986b23b2774a614a42dfe1 not found: ID does not exist" Jul 09 17:38:39.867456 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.867407 2586 scope.go:117] "RemoveContainer" containerID="e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1" Jul 09 17:38:39.867715 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:38:39.867693 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1\": container with ID starting with e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1 not found: ID does not exist" containerID="e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1" Jul 09 17:38:39.867764 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.867734 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1"} err="failed to get container status \"e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1\": rpc error: code = NotFound desc = could not find container \"e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1\": container with ID starting with e261810c3924bbfbc3b219efeb837cb4c41af041ec4d072dced2d11f6b3363f1 not found: ID does not exist" Jul 09 17:38:39.873419 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.873393 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk"] Jul 09 17:38:39.877236 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:39.877214 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7565f4e1-openshift-default-6d565d796c-thgkk"] Jul 09 17:38:40.673295 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:38:40.673259 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" path="/var/lib/kubelet/pods/b89f6ef0-3092-4aa3-a3f7-0c89c85b6149/volumes" Jul 09 17:39:00.645645 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:00.645615 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:39:00.652816 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:00.652795 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:39:28.997751 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:28.997715 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d"] Jul 09 17:39:28.998358 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:28.998344 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" Jul 09 17:39:28.998433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:28.998364 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" Jul 09 17:39:28.998433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:28.998384 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" containerName="maas-api" Jul 09 17:39:28.998433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:28.998393 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" containerName="maas-api" Jul 09 17:39:28.998627 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:28.998533 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" Jul 09 17:39:28.998627 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:28.998552 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" Jul 09 17:39:28.998627 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:28.998562 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a6d4ba-dd47-4948-8fd2-2bc3e59c966f" containerName="maas-api" Jul 09 17:39:29.001837 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.001815 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.004863 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.004839 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-dd4cf6-openshift-default-dockercfg-9j6bg\"" Jul 09 17:39:29.012507 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.012476 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d"] Jul 09 17:39:29.041090 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041054 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-socket\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.041293 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041108 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfced16d-b2bc-45d3-9c40-285d22281127-istio-podinfo\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.041293 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041211 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-credential-socket\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.041293 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041246 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-data\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.041431 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041316 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-envoy\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.041431 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041355 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl9vc\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-kube-api-access-wl9vc\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.041431 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041391 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-istio-token\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.041431 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041430 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-certs\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.041629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.041507 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfced16d-b2bc-45d3-9c40-285d22281127-istiod-ca-cert\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142310 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142269 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-istio-token\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142324 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-certs\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142361 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfced16d-b2bc-45d3-9c40-285d22281127-istiod-ca-cert\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142382 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-socket\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142416 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfced16d-b2bc-45d3-9c40-285d22281127-istio-podinfo\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142489 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-credential-socket\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142549 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142514 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-data\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142891 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142636 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-envoy\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142891 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142691 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vc\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-kube-api-access-wl9vc\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.142976 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.142931 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-socket\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.143124 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.143103 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-credential-socket\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.143209 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.143146 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-data\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.143209 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.143168 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-certs\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.143286 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.143203 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfced16d-b2bc-45d3-9c40-285d22281127-istiod-ca-cert\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.145031 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.145001 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-envoy\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.145189 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.145172 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfced16d-b2bc-45d3-9c40-285d22281127-istio-podinfo\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.152524 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.152496 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-istio-token\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.152652 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.152634 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9vc\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-kube-api-access-wl9vc\") pod \"e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.313802 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.313693 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:29.466739 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.466715 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d"] Jul 09 17:39:29.468853 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:39:29.468824 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfced16d_b2bc_45d3_9c40_285d22281127.slice/crio-fcce47ea9cc3d38aba6cab692c686349eee923853f067224e5778e45c1edd349 WatchSource:0}: Error finding container fcce47ea9cc3d38aba6cab692c686349eee923853f067224e5778e45c1edd349: Status 404 returned error can't find the container with id fcce47ea9cc3d38aba6cab692c686349eee923853f067224e5778e45c1edd349 Jul 09 17:39:29.471662 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.471627 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:29.471782 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.471714 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:29.471782 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:29.471760 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:30.104210 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:30.104171 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" event={"ID":"dfced16d-b2bc-45d3-9c40-285d22281127","Type":"ContainerStarted","Data":"a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f"} Jul 09 17:39:30.104736 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:30.104223 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" event={"ID":"dfced16d-b2bc-45d3-9c40-285d22281127","Type":"ContainerStarted","Data":"fcce47ea9cc3d38aba6cab692c686349eee923853f067224e5778e45c1edd349"} Jul 09 17:39:30.129655 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:30.129600 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" podStartSLOduration=2.129578681 podStartE2EDuration="2.129578681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:39:30.127983473 +0000 UTC m=+1830.057428988" watchObservedRunningTime="2026-07-09 17:39:30.129578681 +0000 UTC m=+1830.059024193" Jul 09 17:39:30.314363 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:30.314319 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:30.319480 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:30.319420 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:31.116326 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:31.116289 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:31.117373 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:31.117354 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:32.135764 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.135720 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v"] Jul 09 17:39:32.136260 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.136233 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" Jul 09 17:39:32.136260 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.136248 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="b89f6ef0-3092-4aa3-a3f7-0c89c85b6149" containerName="istio-proxy" Jul 09 17:39:32.139601 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.139578 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.142849 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.142826 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-dd4cf6-serving-cert\"" Jul 09 17:39:32.150119 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.150083 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v"] Jul 09 17:39:32.173808 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.173777 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64shk\" (UniqueName: \"kubernetes.io/projected/16e9c6b9-01ac-446f-bf68-231121f4e683-kube-api-access-64shk\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.173987 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.173874 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e9c6b9-01ac-446f-bf68-231121f4e683-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.173987 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.173911 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e9c6b9-01ac-446f-bf68-231121f4e683-maas-api-tls\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.274657 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.274617 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64shk\" (UniqueName: \"kubernetes.io/projected/16e9c6b9-01ac-446f-bf68-231121f4e683-kube-api-access-64shk\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.274862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.274708 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e9c6b9-01ac-446f-bf68-231121f4e683-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.274862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.274739 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e9c6b9-01ac-446f-bf68-231121f4e683-maas-api-tls\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.275477 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.275430 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e9c6b9-01ac-446f-bf68-231121f4e683-openshift-service-ca\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.277975 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.277923 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e9c6b9-01ac-446f-bf68-231121f4e683-maas-api-tls\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.285340 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.285306 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64shk\" (UniqueName: \"kubernetes.io/projected/16e9c6b9-01ac-446f-bf68-231121f4e683-kube-api-access-64shk\") pod \"maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.452447 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.452341 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:32.595314 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.595288 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v"] Jul 09 17:39:32.597096 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:39:32.597069 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e9c6b9_01ac_446f_bf68_231121f4e683.slice/crio-a904f2372e08b07e21db7a920e54c3d045df791b12df9b0bc2c3a23e6174807d WatchSource:0}: Error finding container a904f2372e08b07e21db7a920e54c3d045df791b12df9b0bc2c3a23e6174807d: Status 404 returned error can't find the container with id a904f2372e08b07e21db7a920e54c3d045df791b12df9b0bc2c3a23e6174807d Jul 09 17:39:32.598518 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:32.598499 2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:39:33.127844 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:33.127803 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" event={"ID":"16e9c6b9-01ac-446f-bf68-231121f4e683","Type":"ContainerStarted","Data":"a904f2372e08b07e21db7a920e54c3d045df791b12df9b0bc2c3a23e6174807d"} Jul 09 17:39:34.134622 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:34.134586 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" event={"ID":"16e9c6b9-01ac-446f-bf68-231121f4e683","Type":"ContainerStarted","Data":"f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca"} Jul 09 17:39:34.135060 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:34.134695 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:34.156332 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:34.156279 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" podStartSLOduration=1.584620623 podStartE2EDuration="2.156261685s" totalImagesPullingTime="571.641062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:39:34.154376458 +0000 UTC m=+1834.083821991" watchObservedRunningTime="2026-07-09 17:39:34.156261685 +0000 UTC m=+1834.085707197" Jul 09 17:39:37.317565 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.317529 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f"] Jul 09 17:39:37.321273 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.321254 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.325407 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.325379 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-bc6ff5-openshift-default-dockercfg-m7tvk\"" Jul 09 17:39:37.334609 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.334578 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f"] Jul 09 17:39:37.426231 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426190 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-credential-socket\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.426453 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426242 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-token\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.426453 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426375 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-envoy\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.426453 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426420 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-certs\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.426641 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426498 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-socket\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.426641 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426522 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4c9708-d014-44d4-b6a3-91089b288cfb-istiod-ca-cert\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.426748 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426651 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-podinfo\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.426748 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426676 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-data\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.426748 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.426704 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnxxz\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-kube-api-access-xnxxz\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.527898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.527861 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-credential-socket\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.527898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.527902 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-token\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.527950 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-envoy\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.527969 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-certs\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.527989 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-socket\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.528003 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4c9708-d014-44d4-b6a3-91089b288cfb-istiod-ca-cert\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.528096 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-podinfo\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.528118 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-data\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.528137 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnxxz\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-kube-api-access-xnxxz\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528550 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.528361 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-credential-socket\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528550 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.528398 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-certs\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528662 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.528556 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-socket\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.528764 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.528738 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-data\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.529077 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.529050 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4c9708-d014-44d4-b6a3-91089b288cfb-istiod-ca-cert\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.530420 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.530400 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-envoy\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.530754 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.530734 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-podinfo\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.539109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.539083 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnxxz\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-kube-api-access-xnxxz\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.539250 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.539162 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-token\") pod \"e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.633876 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.633780 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:37.787195 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.787164 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f"] Jul 09 17:39:37.788771 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:39:37.788742 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4c9708_d014_44d4_b6a3_91089b288cfb.slice/crio-9f50f8c3a9031abb619586a4ee5d8c91ba176259a44fec4bf2bf3cd9ede34317 WatchSource:0}: Error finding container 9f50f8c3a9031abb619586a4ee5d8c91ba176259a44fec4bf2bf3cd9ede34317: Status 404 returned error can't find the container with id 9f50f8c3a9031abb619586a4ee5d8c91ba176259a44fec4bf2bf3cd9ede34317 Jul 09 17:39:37.791125 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.791091 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:37.791192 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.791163 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:37.791230 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:37.791191 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:38.157770 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:38.157732 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" event={"ID":"cd4c9708-d014-44d4-b6a3-91089b288cfb","Type":"ContainerStarted","Data":"d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652"} Jul 09 17:39:38.157943 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:38.157800 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" event={"ID":"cd4c9708-d014-44d4-b6a3-91089b288cfb","Type":"ContainerStarted","Data":"9f50f8c3a9031abb619586a4ee5d8c91ba176259a44fec4bf2bf3cd9ede34317"} Jul 09 17:39:38.184084 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:38.184034 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" podStartSLOduration=1.184017324 podStartE2EDuration="1.184017324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:39:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:39:38.18163446 +0000 UTC m=+1838.111080047" watchObservedRunningTime="2026-07-09 17:39:38.184017324 +0000 UTC m=+1838.113462916" Jul 09 17:39:38.634705 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:38.634665 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:38.640143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:38.640115 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:39.164723 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:39.164689 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:39.165644 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:39.165621 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:40.145096 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:40.145068 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:39:47.157622 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:47.157579 2586 patch_prober.go:28] interesting pod/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 17:39:47.158069 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:47.157641 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 17:39:49.228768 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:49.228733 2586 generic.go:356] "Generic (PLEG): container finished" podID="dfced16d-b2bc-45d3-9c40-285d22281127" containerID="a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f" exitCode=0 Jul 09 17:39:49.229226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:49.228813 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" event={"ID":"dfced16d-b2bc-45d3-9c40-285d22281127","Type":"ContainerDied","Data":"a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f"} Jul 09 17:39:49.229311 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:49.229294 2586 scope.go:117] "RemoveContainer" containerID="a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f" Jul 09 17:39:49.230341 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:49.230312 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:49.230505 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:49.230387 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:49.230505 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:49.230425 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:50.236623 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:50.236585 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" event={"ID":"dfced16d-b2bc-45d3-9c40-285d22281127","Type":"ContainerStarted","Data":"bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929"} Jul 09 17:39:50.314193 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:50.314145 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:50.314611 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:50.314576 2586 patch_prober.go:28] interesting pod/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 17:39:50.314737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:50.314648 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 17:39:51.314610 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:51.314561 2586 patch_prober.go:28] interesting pod/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 17:39:51.315146 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:51.314637 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 17:39:52.314534 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:52.314492 2586 patch_prober.go:28] interesting pod/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 17:39:52.314758 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:52.314558 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 17:39:53.315164 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:53.315081 2586 patch_prober.go:28] interesting pod/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 17:39:53.315164 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:53.315142 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 17:39:54.314406 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:54.314360 2586 patch_prober.go:28] interesting pod/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 09 17:39:54.314634 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:54.314453 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 09 17:39:55.206703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:55.206656 2586 patch_prober.go:28] interesting pod/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 17:39:55.207142 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:55.206753 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 17:39:55.318807 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:55.318769 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:55.319093 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:55.319066 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:55.319925 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:55.319900 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:39:56.270681 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:56.270602 2586 generic.go:356] "Generic (PLEG): container finished" podID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerID="d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652" exitCode=0 Jul 09 17:39:56.271101 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:56.270674 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" event={"ID":"cd4c9708-d014-44d4-b6a3-91089b288cfb","Type":"ContainerDied","Data":"d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652"} Jul 09 17:39:56.271241 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:56.271226 2586 scope.go:117] "RemoveContainer" containerID="d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652" Jul 09 17:39:56.272258 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:56.272232 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:56.272330 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:56.272313 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:56.272380 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:56.272364 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:39:57.277504 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:57.277466 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" event={"ID":"cd4c9708-d014-44d4-b6a3-91089b288cfb","Type":"ContainerStarted","Data":"3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b"} Jul 09 17:39:57.634595 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:57.634495 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:39:57.634807 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:57.634784 2586 patch_prober.go:28] interesting pod/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 17:39:57.634866 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:57.634845 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 17:39:58.634851 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:58.634809 2586 patch_prober.go:28] interesting pod/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 17:39:58.635346 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:58.634877 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 17:39:59.634718 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:59.634679 2586 patch_prober.go:28] interesting pod/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 17:39:59.634941 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:39:59.634746 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 17:40:00.634900 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:00.634859 2586 patch_prober.go:28] interesting pod/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 09 17:40:00.635107 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:00.634946 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 09 17:40:01.637953 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:01.637925 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:40:01.638348 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:01.638186 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:40:01.638977 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:01.638955 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:40:35.376166 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.376125 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v"] Jul 09 17:40:35.376964 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.376904 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" podUID="16e9c6b9-01ac-446f-bf68-231121f4e683" containerName="maas-api" containerID="cri-o://f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca" gracePeriod=30 Jul 09 17:40:35.634114 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.634046 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:40:35.747070 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.747033 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e9c6b9-01ac-446f-bf68-231121f4e683-openshift-service-ca\") pod \"16e9c6b9-01ac-446f-bf68-231121f4e683\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " Jul 09 17:40:35.747292 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.747260 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64shk\" (UniqueName: \"kubernetes.io/projected/16e9c6b9-01ac-446f-bf68-231121f4e683-kube-api-access-64shk\") pod \"16e9c6b9-01ac-446f-bf68-231121f4e683\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " Jul 09 17:40:35.747369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.747295 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e9c6b9-01ac-446f-bf68-231121f4e683-maas-api-tls\") pod \"16e9c6b9-01ac-446f-bf68-231121f4e683\" (UID: \"16e9c6b9-01ac-446f-bf68-231121f4e683\") " Jul 09 17:40:35.747545 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.747516 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16e9c6b9-01ac-446f-bf68-231121f4e683-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16e9c6b9-01ac-446f-bf68-231121f4e683" (UID: "16e9c6b9-01ac-446f-bf68-231121f4e683"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:40:35.747795 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.747764 2586 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16e9c6b9-01ac-446f-bf68-231121f4e683-openshift-service-ca\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:40:35.749417 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.749395 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16e9c6b9-01ac-446f-bf68-231121f4e683-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16e9c6b9-01ac-446f-bf68-231121f4e683" (UID: "16e9c6b9-01ac-446f-bf68-231121f4e683"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:40:35.749492 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.749475 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16e9c6b9-01ac-446f-bf68-231121f4e683-kube-api-access-64shk" (OuterVolumeSpecName: "kube-api-access-64shk") pod "16e9c6b9-01ac-446f-bf68-231121f4e683" (UID: "16e9c6b9-01ac-446f-bf68-231121f4e683"). InnerVolumeSpecName "kube-api-access-64shk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:40:35.849078 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.849031 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64shk\" (UniqueName: \"kubernetes.io/projected/16e9c6b9-01ac-446f-bf68-231121f4e683-kube-api-access-64shk\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:40:35.849078 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:35.849070 2586 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16e9c6b9-01ac-446f-bf68-231121f4e683-maas-api-tls\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:40:36.498617 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.498580 2586 generic.go:356] "Generic (PLEG): container finished" podID="16e9c6b9-01ac-446f-bf68-231121f4e683" containerID="f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca" exitCode=0 Jul 09 17:40:36.498617 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.498619 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" Jul 09 17:40:36.499203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.498655 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" event={"ID":"16e9c6b9-01ac-446f-bf68-231121f4e683","Type":"ContainerDied","Data":"f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca"} Jul 09 17:40:36.499203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.498716 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v" event={"ID":"16e9c6b9-01ac-446f-bf68-231121f4e683","Type":"ContainerDied","Data":"a904f2372e08b07e21db7a920e54c3d045df791b12df9b0bc2c3a23e6174807d"} Jul 09 17:40:36.499203 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.498740 2586 scope.go:117] "RemoveContainer" containerID="f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca" Jul 09 17:40:36.510984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.510962 2586 scope.go:117] "RemoveContainer" containerID="f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca" Jul 09 17:40:36.511266 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:40:36.511245 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca\": container with ID starting with f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca not found: ID does not exist" containerID="f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca" Jul 09 17:40:36.511338 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.511275 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca"} err="failed to get container status \"f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca\": rpc error: code = NotFound desc = could not find container \"f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca\": container with ID starting with f711121f14edcf99711fcb722f87304f5c56bed005967c802fd163aeb3c2c5ca not found: ID does not exist" Jul 09 17:40:36.522110 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.522083 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v"] Jul 09 17:40:36.528165 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.528138 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-dd4cf6-57bccf7fbf-6zl8v"] Jul 09 17:40:36.531222 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:40:36.531202 2586 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e9c6b9_01ac_446f_bf68_231121f4e683.slice/crio-a904f2372e08b07e21db7a920e54c3d045df791b12df9b0bc2c3a23e6174807d\": RecentStats: unable to find data in memory cache]" Jul 09 17:40:36.672225 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:40:36.672189 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16e9c6b9-01ac-446f-bf68-231121f4e683" path="/var/lib/kubelet/pods/16e9c6b9-01ac-446f-bf68-231121f4e683/volumes" Jul 09 17:41:17.102312 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:17.102274 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d"] Jul 09 17:41:17.102914 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:17.102557 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" containerID="cri-o://bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929" gracePeriod=30 Jul 09 17:41:22.371204 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.371126 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:41:22.426862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.426829 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-certs\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427054 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.426907 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-istio-token\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427054 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.426936 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfced16d-b2bc-45d3-9c40-285d22281127-istio-podinfo\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427054 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.426985 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl9vc\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-kube-api-access-wl9vc\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427233 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427142 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-socket\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427233 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427149 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:22.427233 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427183 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfced16d-b2bc-45d3-9c40-285d22281127-istiod-ca-cert\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427233 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427220 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-credential-socket\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427464 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427246 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-envoy\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427464 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427341 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-data\") pod \"dfced16d-b2bc-45d3-9c40-285d22281127\" (UID: \"dfced16d-b2bc-45d3-9c40-285d22281127\") " Jul 09 17:41:22.427464 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427387 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:22.427732 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427654 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:22.427732 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427580 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfced16d-b2bc-45d3-9c40-285d22281127-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:41:22.427956 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427929 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.427956 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427952 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dfced16d-b2bc-45d3-9c40-285d22281127-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.428065 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427969 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.428065 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.427982 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.428622 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.428597 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-data" (OuterVolumeSpecName: "istio-data") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:22.429574 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.429544 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-kube-api-access-wl9vc" (OuterVolumeSpecName: "kube-api-access-wl9vc") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "kube-api-access-wl9vc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:41:22.429772 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.429744 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-istio-token" (OuterVolumeSpecName: "istio-token") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:41:22.429885 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.429832 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dfced16d-b2bc-45d3-9c40-285d22281127-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:41:22.429934 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.429889 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dfced16d-b2bc-45d3-9c40-285d22281127" (UID: "dfced16d-b2bc-45d3-9c40-285d22281127"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:22.529280 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.529242 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl9vc\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-kube-api-access-wl9vc\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.529280 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.529273 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.529280 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.529282 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dfced16d-b2bc-45d3-9c40-285d22281127-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.529280 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.529292 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dfced16d-b2bc-45d3-9c40-285d22281127-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.529614 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.529302 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dfced16d-b2bc-45d3-9c40-285d22281127-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:22.750131 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.750092 2586 generic.go:356] "Generic (PLEG): container finished" podID="dfced16d-b2bc-45d3-9c40-285d22281127" containerID="bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929" exitCode=0 Jul 09 17:41:22.750344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.750142 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" Jul 09 17:41:22.750344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.750134 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" event={"ID":"dfced16d-b2bc-45d3-9c40-285d22281127","Type":"ContainerDied","Data":"bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929"} Jul 09 17:41:22.750344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.750216 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d" event={"ID":"dfced16d-b2bc-45d3-9c40-285d22281127","Type":"ContainerDied","Data":"fcce47ea9cc3d38aba6cab692c686349eee923853f067224e5778e45c1edd349"} Jul 09 17:41:22.750344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.750246 2586 scope.go:117] "RemoveContainer" containerID="bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929" Jul 09 17:41:22.759358 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.759292 2586 scope.go:117] "RemoveContainer" containerID="a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f" Jul 09 17:41:22.767645 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.767623 2586 scope.go:117] "RemoveContainer" containerID="bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929" Jul 09 17:41:22.767923 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:41:22.767895 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929\": container with ID starting with bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929 not found: ID does not exist" containerID="bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929" Jul 09 17:41:22.768042 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.767935 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929"} err="failed to get container status \"bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929\": rpc error: code = NotFound desc = could not find container \"bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929\": container with ID starting with bc6316c7ae5714e857ac665aa696ef18788bf227fbc1402b736758eab95e2929 not found: ID does not exist" Jul 09 17:41:22.768042 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.767954 2586 scope.go:117] "RemoveContainer" containerID="a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f" Jul 09 17:41:22.768269 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:41:22.768250 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f\": container with ID starting with a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f not found: ID does not exist" containerID="a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f" Jul 09 17:41:22.768452 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.768278 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f"} err="failed to get container status \"a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f\": rpc error: code = NotFound desc = could not find container \"a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f\": container with ID starting with a1779e39760c37870efe351fba56fb49babd0aded0cca2c1cb7537bfadb85f1f not found: ID does not exist" Jul 09 17:41:22.771782 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.771759 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d"] Jul 09 17:41:22.773798 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:22.773777 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4cf6-openshift-default-6b868bf8f5-kzk6d"] Jul 09 17:41:24.055677 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:24.055644 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f"] Jul 09 17:41:24.056171 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:24.055897 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" containerID="cri-o://3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b" gracePeriod=30 Jul 09 17:41:24.672542 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:24.672505 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" path="/var/lib/kubelet/pods/dfced16d-b2bc-45d3-9c40-285d22281127/volumes" Jul 09 17:41:25.069994 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.069953 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9"] Jul 09 17:41:25.070629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.070605 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" Jul 09 17:41:25.070629 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.070629 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" Jul 09 17:41:25.070805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.070650 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16e9c6b9-01ac-446f-bf68-231121f4e683" containerName="maas-api" Jul 09 17:41:25.070805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.070659 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="16e9c6b9-01ac-446f-bf68-231121f4e683" containerName="maas-api" Jul 09 17:41:25.070805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.070798 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="16e9c6b9-01ac-446f-bf68-231121f4e683" containerName="maas-api" Jul 09 17:41:25.070964 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.070812 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" Jul 09 17:41:25.070964 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.070826 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" Jul 09 17:41:25.075948 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.075927 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.078945 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.078922 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-beee07-openshift-default-dockercfg-slkxv\"" Jul 09 17:41:25.084992 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.084967 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9"] Jul 09 17:41:25.156173 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156133 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-certs\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.156357 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156249 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-socket\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.156357 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156295 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5jkw\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-kube-api-access-j5jkw\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.156357 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156323 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60f0a69-67d0-4da3-87cc-fe346359fb82-istiod-ca-cert\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.156511 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156362 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-envoy\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.156511 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156416 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-credential-socket\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.156511 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156482 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-token\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.156653 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156501 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-podinfo\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.156653 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.156545 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-data\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257414 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257367 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-envoy\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257414 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257424 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-credential-socket\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257666 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257625 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-token\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257704 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257664 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-podinfo\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257746 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257700 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-data\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257746 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257734 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-certs\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257837 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257802 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-credential-socket\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257889 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257859 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-socket\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.257983 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.257945 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5jkw\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-kube-api-access-j5jkw\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.258053 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.258015 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60f0a69-67d0-4da3-87cc-fe346359fb82-istiod-ca-cert\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.258122 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.258067 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-data\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.258214 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.258181 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-socket\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.258271 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.258259 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-certs\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.258670 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.258648 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60f0a69-67d0-4da3-87cc-fe346359fb82-istiod-ca-cert\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.259876 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.259853 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-envoy\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.260092 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.260074 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-podinfo\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.266121 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.266095 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-token\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.266224 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.266160 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5jkw\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-kube-api-access-j5jkw\") pod \"e2e-api-a-beee07-openshift-default-68fd676879-bdwg9\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.388317 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.388224 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:25.535984 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.535955 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9"] Jul 09 17:41:25.538627 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:41:25.538592 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode60f0a69_67d0_4da3_87cc_fe346359fb82.slice/crio-00d0084ab2ec59f265d75d5a28743f8f7cb869053354c7c93f1b9b37a0c51b31 WatchSource:0}: Error finding container 00d0084ab2ec59f265d75d5a28743f8f7cb869053354c7c93f1b9b37a0c51b31: Status 404 returned error can't find the container with id 00d0084ab2ec59f265d75d5a28743f8f7cb869053354c7c93f1b9b37a0c51b31 Jul 09 17:41:25.540946 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.540914 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:25.541051 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.540997 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:25.549142 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.541525 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:25.766234 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.766200 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" event={"ID":"e60f0a69-67d0-4da3-87cc-fe346359fb82","Type":"ContainerStarted","Data":"45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395"} Jul 09 17:41:25.766424 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.766259 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" event={"ID":"e60f0a69-67d0-4da3-87cc-fe346359fb82","Type":"ContainerStarted","Data":"00d0084ab2ec59f265d75d5a28743f8f7cb869053354c7c93f1b9b37a0c51b31"} Jul 09 17:41:25.788369 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:25.788311 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" podStartSLOduration=0.788289733 podStartE2EDuration="788.289733ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:25.787253243 +0000 UTC m=+1945.716698755" watchObservedRunningTime="2026-07-09 17:41:25.788289733 +0000 UTC m=+1945.717735250" Jul 09 17:41:26.388541 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:26.388507 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:26.394077 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:26.394050 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:26.771672 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:26.771635 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:26.772715 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:26.772692 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:29.319734 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.319669 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:41:29.400036 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400002 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-envoy\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400219 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400079 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-socket\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400219 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400118 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnxxz\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-kube-api-access-xnxxz\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400219 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400171 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4c9708-d014-44d4-b6a3-91089b288cfb-istiod-ca-cert\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400351 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400218 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-podinfo\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400351 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400266 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-certs\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400351 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400291 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-token\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400351 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400325 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-credential-socket\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400596 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400379 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-data\") pod \"cd4c9708-d014-44d4-b6a3-91089b288cfb\" (UID: \"cd4c9708-d014-44d4-b6a3-91089b288cfb\") " Jul 09 17:41:29.400596 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400407 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:29.400715 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400592 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd4c9708-d014-44d4-b6a3-91089b288cfb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:41:29.400715 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400676 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:29.400715 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400597 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:29.400993 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400906 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4c9708-d014-44d4-b6a3-91089b288cfb-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.400993 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400941 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.400993 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400955 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.400993 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.400969 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.401426 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.401400 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:29.402532 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.402490 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:41:29.402532 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.402518 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:41:29.402657 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.402603 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:41:29.402694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.402660 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-kube-api-access-xnxxz" (OuterVolumeSpecName: "kube-api-access-xnxxz") pod "cd4c9708-d014-44d4-b6a3-91089b288cfb" (UID: "cd4c9708-d014-44d4-b6a3-91089b288cfb"). InnerVolumeSpecName "kube-api-access-xnxxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:41:29.501691 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.501651 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnxxz\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-kube-api-access-xnxxz\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.501691 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.501684 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.501691 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.501695 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.501691 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.501703 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.501975 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.501712 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4c9708-d014-44d4-b6a3-91089b288cfb-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:41:29.788235 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.788198 2586 generic.go:356] "Generic (PLEG): container finished" podID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerID="3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b" exitCode=0 Jul 09 17:41:29.788465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.788244 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" Jul 09 17:41:29.788465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.788286 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" event={"ID":"cd4c9708-d014-44d4-b6a3-91089b288cfb","Type":"ContainerDied","Data":"3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b"} Jul 09 17:41:29.788465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.788351 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f" event={"ID":"cd4c9708-d014-44d4-b6a3-91089b288cfb","Type":"ContainerDied","Data":"9f50f8c3a9031abb619586a4ee5d8c91ba176259a44fec4bf2bf3cd9ede34317"} Jul 09 17:41:29.788465 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.788382 2586 scope.go:117] "RemoveContainer" containerID="3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b" Jul 09 17:41:29.798328 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.798307 2586 scope.go:117] "RemoveContainer" containerID="d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652" Jul 09 17:41:29.806736 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.806718 2586 scope.go:117] "RemoveContainer" containerID="3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b" Jul 09 17:41:29.807004 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:41:29.806981 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b\": container with ID starting with 3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b not found: ID does not exist" containerID="3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b" Jul 09 17:41:29.807086 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.807017 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b"} err="failed to get container status \"3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b\": rpc error: code = NotFound desc = could not find container \"3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b\": container with ID starting with 3f1659cf470606285b24422b5b5e79d212c165ec5d7b22b08e42a592c15c019b not found: ID does not exist" Jul 09 17:41:29.807086 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.807042 2586 scope.go:117] "RemoveContainer" containerID="d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652" Jul 09 17:41:29.807290 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:41:29.807274 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652\": container with ID starting with d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652 not found: ID does not exist" containerID="d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652" Jul 09 17:41:29.807330 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.807296 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652"} err="failed to get container status \"d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652\": rpc error: code = NotFound desc = could not find container \"d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652\": container with ID starting with d83c94871836f36a46060be89b9ddc6a865f0a788cf9c287a2bcb7d951853652 not found: ID does not exist" Jul 09 17:41:29.818822 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.818793 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f"] Jul 09 17:41:29.823993 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:29.823961 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-bc6ff5-openshift-default-5966568cd7-p2b2f"] Jul 09 17:41:30.678585 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:30.678545 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" path="/var/lib/kubelet/pods/cd4c9708-d014-44d4-b6a3-91089b288cfb/volumes" Jul 09 17:41:33.071290 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.071222 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf"] Jul 09 17:41:33.071892 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.071873 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" Jul 09 17:41:33.071892 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.071893 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfced16d-b2bc-45d3-9c40-285d22281127" containerName="istio-proxy" Jul 09 17:41:33.072017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.071902 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" Jul 09 17:41:33.072017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.071908 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" Jul 09 17:41:33.072017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.071920 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" Jul 09 17:41:33.072017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.071931 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" Jul 09 17:41:33.072017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.071989 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" Jul 09 17:41:33.074895 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.074875 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.078037 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.078013 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b6bea1-openshift-default-dockercfg-n2q88\"" Jul 09 17:41:33.087792 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.087763 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf"] Jul 09 17:41:33.140520 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140484 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-certs\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.140737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140564 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istiod-ca-cert\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.140737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140600 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-data\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.140737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140656 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shwvz\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-kube-api-access-shwvz\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.140737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140698 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-token\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.141030 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140755 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-credential-socket\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.141030 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140811 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-socket\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.141030 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140901 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-podinfo\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.141030 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.140950 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-envoy\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.241909 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.241874 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-podinfo\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.241929 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-envoy\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.241965 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-certs\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242020 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istiod-ca-cert\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242053 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-data\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242084 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shwvz\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-kube-api-access-shwvz\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242116 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-token\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242156 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-credential-socket\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242210 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-socket\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242590 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242570 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-certs\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242700 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242661 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-data\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242861 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242769 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-credential-socket\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242932 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242915 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-socket\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.242989 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.242942 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istiod-ca-cert\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.244378 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.244358 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-envoy\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.244747 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.244726 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-podinfo\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.250913 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.250870 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-token\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.251044 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.250984 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shwvz\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-kube-api-access-shwvz\") pod \"e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.386317 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.386227 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:33.533751 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.533725 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf"] Jul 09 17:41:33.535927 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:41:33.535888 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbca4bd_0e73_49bf_b821_dfa23699e4a5.slice/crio-3841544d888fdd3c658a2eb1664be0b8b8a9858e4326a32801c8513ff96f2711 WatchSource:0}: Error finding container 3841544d888fdd3c658a2eb1664be0b8b8a9858e4326a32801c8513ff96f2711: Status 404 returned error can't find the container with id 3841544d888fdd3c658a2eb1664be0b8b8a9858e4326a32801c8513ff96f2711 Jul 09 17:41:33.538185 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.538147 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:33.538289 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.538220 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:33.538289 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.538280 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:33.815374 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.815337 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" event={"ID":"8fbca4bd-0e73-49bf-b821-dfa23699e4a5","Type":"ContainerStarted","Data":"02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38"} Jul 09 17:41:33.815576 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.815395 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" event={"ID":"8fbca4bd-0e73-49bf-b821-dfa23699e4a5","Type":"ContainerStarted","Data":"3841544d888fdd3c658a2eb1664be0b8b8a9858e4326a32801c8513ff96f2711"} Jul 09 17:41:33.840226 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:33.840170 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" podStartSLOduration=0.840152496 podStartE2EDuration="840.152496ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:41:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:41:33.836989134 +0000 UTC m=+1953.766434650" watchObservedRunningTime="2026-07-09 17:41:33.840152496 +0000 UTC m=+1953.769598007" Jul 09 17:41:34.387195 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:34.387160 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:34.401116 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:34.401069 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:34.820667 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:34.820634 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:34.821705 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:34.821685 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:40.854870 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:40.854782 2586 generic.go:356] "Generic (PLEG): container finished" podID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerID="45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395" exitCode=0 Jul 09 17:41:40.854870 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:40.854851 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" event={"ID":"e60f0a69-67d0-4da3-87cc-fe346359fb82","Type":"ContainerDied","Data":"45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395"} Jul 09 17:41:40.855308 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:40.855257 2586 scope.go:117] "RemoveContainer" containerID="45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395" Jul 09 17:41:40.856296 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:40.856269 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:40.856349 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:40.856339 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:40.856387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:40.856375 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:41.771707 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:41.771671 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:41.862672 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:41.862637 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" event={"ID":"e60f0a69-67d0-4da3-87cc-fe346359fb82","Type":"ContainerStarted","Data":"a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a"} Jul 09 17:41:42.389038 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:42.388978 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:42.389786 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:42.389758 2586 patch_prober.go:28] interesting pod/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 17:41:42.390219 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:42.390182 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 17:41:43.388898 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:43.388848 2586 patch_prober.go:28] interesting pod/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 17:41:43.389434 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:43.388936 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 17:41:44.389390 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:44.388880 2586 patch_prober.go:28] interesting pod/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 17:41:44.389390 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:44.388954 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 17:41:45.389484 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:45.389415 2586 patch_prober.go:28] interesting pod/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 09 17:41:45.390023 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:45.389511 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 09 17:41:46.398267 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:46.398226 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:46.398804 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:46.398664 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:46.403924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:46.403880 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:41:50.862663 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:50.862611 2586 patch_prober.go:28] interesting pod/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 17:41:50.863233 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:50.862693 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 17:41:51.930919 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:51.930879 2586 generic.go:356] "Generic (PLEG): container finished" podID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerID="02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38" exitCode=0 Jul 09 17:41:51.931426 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:51.930955 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" event={"ID":"8fbca4bd-0e73-49bf-b821-dfa23699e4a5","Type":"ContainerDied","Data":"02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38"} Jul 09 17:41:51.931540 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:51.931426 2586 scope.go:117] "RemoveContainer" containerID="02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38" Jul 09 17:41:51.932588 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:51.932560 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:51.932694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:51.932639 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:51.932694 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:51.932672 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:41:52.937641 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:52.937602 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" event={"ID":"8fbca4bd-0e73-49bf-b821-dfa23699e4a5","Type":"ContainerStarted","Data":"b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0"} Jul 09 17:41:53.387623 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:53.387580 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:53.388131 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:53.388100 2586 patch_prober.go:28] interesting pod/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 17:41:53.388227 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:53.388166 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 17:41:54.387028 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:54.386981 2586 patch_prober.go:28] interesting pod/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 17:41:54.387582 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:54.387059 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 17:41:55.102817 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:55.102773 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9"] Jul 09 17:41:55.103064 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:55.103028 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" containerID="cri-o://a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a" gracePeriod=30 Jul 09 17:41:55.387665 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:55.387564 2586 patch_prober.go:28] interesting pod/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 17:41:55.388271 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:55.388234 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 17:41:56.392608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:56.392559 2586 patch_prober.go:28] interesting pod/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 09 17:41:56.393151 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:56.392635 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 09 17:41:57.390741 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:57.390709 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:57.391014 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:57.390985 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:41:57.391761 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:41:57.391742 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:42:00.373282 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.373211 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:42:00.453801 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.453760 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5jkw\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-kube-api-access-j5jkw\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454002 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.453822 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-token\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454002 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.453866 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60f0a69-67d0-4da3-87cc-fe346359fb82-istiod-ca-cert\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454002 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.453922 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-certs\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454002 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.453967 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-credential-socket\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454227 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454065 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-podinfo\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454227 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454089 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-data\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454227 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454119 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-socket\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454227 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454166 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-envoy\") pod \"e60f0a69-67d0-4da3-87cc-fe346359fb82\" (UID: \"e60f0a69-67d0-4da3-87cc-fe346359fb82\") " Jul 09 17:42:00.454420 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454220 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:00.454420 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454327 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:00.454420 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454344 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e60f0a69-67d0-4da3-87cc-fe346359fb82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:42:00.454643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454495 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:00.454643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454615 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.454643 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454637 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e60f0a69-67d0-4da3-87cc-fe346359fb82-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.454813 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454651 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.454813 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.454663 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.455341 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.455312 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-data" (OuterVolumeSpecName: "istio-data") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:00.456241 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.456206 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-token" (OuterVolumeSpecName: "istio-token") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:42:00.456241 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.456232 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-kube-api-access-j5jkw" (OuterVolumeSpecName: "kube-api-access-j5jkw") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "kube-api-access-j5jkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:42:00.456392 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.456290 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:00.456392 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.456325 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e60f0a69-67d0-4da3-87cc-fe346359fb82" (UID: "e60f0a69-67d0-4da3-87cc-fe346359fb82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:42:00.555387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.555346 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.555387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.555376 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.555387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.555385 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.555387 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.555394 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5jkw\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-kube-api-access-j5jkw\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.555696 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.555402 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e60f0a69-67d0-4da3-87cc-fe346359fb82-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:00.981225 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.981183 2586 generic.go:356] "Generic (PLEG): container finished" podID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerID="a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a" exitCode=0 Jul 09 17:42:00.981406 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.981238 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" event={"ID":"e60f0a69-67d0-4da3-87cc-fe346359fb82","Type":"ContainerDied","Data":"a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a"} Jul 09 17:42:00.981406 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.981253 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" Jul 09 17:42:00.981406 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.981297 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9" event={"ID":"e60f0a69-67d0-4da3-87cc-fe346359fb82","Type":"ContainerDied","Data":"00d0084ab2ec59f265d75d5a28743f8f7cb869053354c7c93f1b9b37a0c51b31"} Jul 09 17:42:00.981406 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.981322 2586 scope.go:117] "RemoveContainer" containerID="a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a" Jul 09 17:42:00.989960 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.989939 2586 scope.go:117] "RemoveContainer" containerID="45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395" Jul 09 17:42:00.999073 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.998246 2586 scope.go:117] "RemoveContainer" containerID="a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a" Jul 09 17:42:00.999272 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:42:00.999240 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a\": container with ID starting with a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a not found: ID does not exist" containerID="a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a" Jul 09 17:42:00.999398 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.999284 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a"} err="failed to get container status \"a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a\": rpc error: code = NotFound desc = could not find container \"a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a\": container with ID starting with a761b8a6ef1b36c400021860a46ac83b96a214a0cdbf34d13b8dc5aeb642d77a not found: ID does not exist" Jul 09 17:42:00.999398 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.999310 2586 scope.go:117] "RemoveContainer" containerID="45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395" Jul 09 17:42:00.999684 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:42:00.999659 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395\": container with ID starting with 45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395 not found: ID does not exist" containerID="45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395" Jul 09 17:42:00.999761 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:00.999689 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395"} err="failed to get container status \"45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395\": rpc error: code = NotFound desc = could not find container \"45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395\": container with ID starting with 45612681f66458280c69c9e42f3950ecddf58383de92827ecaff3dacdf903395 not found: ID does not exist" Jul 09 17:42:01.001327 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:01.001306 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9"] Jul 09 17:42:01.005046 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:01.005024 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-beee07-openshift-default-68fd676879-bdwg9"] Jul 09 17:42:02.091773 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:02.091726 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf"] Jul 09 17:42:02.092278 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:02.091970 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" containerID="cri-o://b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0" gracePeriod=30 Jul 09 17:42:02.671924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:02.671888 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" path="/var/lib/kubelet/pods/e60f0a69-67d0-4da3-87cc-fe346359fb82/volumes" Jul 09 17:42:03.009786 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.009743 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v"] Jul 09 17:42:03.010405 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.010385 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" Jul 09 17:42:03.010405 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.010405 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" Jul 09 17:42:03.010636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.010431 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" Jul 09 17:42:03.010636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.010455 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" Jul 09 17:42:03.010636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.010555 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" Jul 09 17:42:03.010636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.010570 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd4c9708-d014-44d4-b6a3-91089b288cfb" containerName="istio-proxy" Jul 09 17:42:03.010636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.010582 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="e60f0a69-67d0-4da3-87cc-fe346359fb82" containerName="istio-proxy" Jul 09 17:42:03.015578 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.015559 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.018567 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.018543 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-77a679-openshift-default-dockercfg-r8l6r\"" Jul 09 17:42:03.023718 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.023325 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v"] Jul 09 17:42:03.082978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.082935 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-credential-socket\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.082978 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.082976 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-istio-token\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.083230 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.082997 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-socket\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.083230 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.083084 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-envoy\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.083230 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.083114 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-data\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.083230 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.083187 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b972c82-e791-4e71-961a-8eda14096bf1-istio-podinfo\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.083385 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.083274 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79ckk\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-kube-api-access-79ckk\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.083385 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.083339 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-certs\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.083385 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.083358 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b972c82-e791-4e71-961a-8eda14096bf1-istiod-ca-cert\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184112 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184073 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-certs\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184112 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184124 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b972c82-e791-4e71-961a-8eda14096bf1-istiod-ca-cert\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184155 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-credential-socket\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184180 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-istio-token\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184208 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-socket\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184247 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-envoy\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184388 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-data\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184503 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b972c82-e791-4e71-961a-8eda14096bf1-istio-podinfo\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184600 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79ckk\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-kube-api-access-79ckk\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184611 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-certs\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.184703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184651 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-credential-socket\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.185155 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.184919 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-socket\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.185155 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.185077 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-data\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.185290 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.185262 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b972c82-e791-4e71-961a-8eda14096bf1-istiod-ca-cert\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.186926 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.186902 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-envoy\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.187171 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.187153 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b972c82-e791-4e71-961a-8eda14096bf1-istio-podinfo\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.193626 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.193592 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-istio-token\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.193784 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.193706 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79ckk\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-kube-api-access-79ckk\") pod \"e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.328135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.328028 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:03.499713 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.499673 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v"] Jul 09 17:42:03.501245 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:42:03.501212 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b972c82_e791_4e71_961a_8eda14096bf1.slice/crio-8bdc485a48f482649841e8d6611a7bdaf184df9479654c7b288b965bbda0060d WatchSource:0}: Error finding container 8bdc485a48f482649841e8d6611a7bdaf184df9479654c7b288b965bbda0060d: Status 404 returned error can't find the container with id 8bdc485a48f482649841e8d6611a7bdaf184df9479654c7b288b965bbda0060d Jul 09 17:42:03.503725 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.503690 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:03.503821 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.503764 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:03.503821 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:03.503796 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:04.001086 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:04.001050 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" event={"ID":"7b972c82-e791-4e71-961a-8eda14096bf1","Type":"ContainerStarted","Data":"631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7"} Jul 09 17:42:04.001271 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:04.001107 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" event={"ID":"7b972c82-e791-4e71-961a-8eda14096bf1","Type":"ContainerStarted","Data":"8bdc485a48f482649841e8d6611a7bdaf184df9479654c7b288b965bbda0060d"} Jul 09 17:42:04.024192 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:04.024142 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" podStartSLOduration=2.024125897 podStartE2EDuration="2.024125897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:42:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:42:04.021343727 +0000 UTC m=+1983.950789252" watchObservedRunningTime="2026-07-09 17:42:04.024125897 +0000 UTC m=+1983.953571408" Jul 09 17:42:04.329170 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:04.329065 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:04.334298 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:04.334267 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:05.006511 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:05.006461 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:05.007671 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:05.007643 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:07.353659 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.353586 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:42:07.430872 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.430836 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-data\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.430872 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.430875 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-token\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.431128 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.430911 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-certs\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.431128 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.430956 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shwvz\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-kube-api-access-shwvz\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.431128 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431032 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-credential-socket\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.431128 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431056 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-podinfo\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.431128 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431080 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-envoy\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.431388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431143 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-socket\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.431388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431184 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istiod-ca-cert\") pod \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\" (UID: \"8fbca4bd-0e73-49bf-b821-dfa23699e4a5\") " Jul 09 17:42:07.431388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431270 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:07.431388 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431315 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:07.431625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431545 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:07.431625 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431609 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:07.431726 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431633 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:07.431726 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431649 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:42:07.431927 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.431902 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-data" (OuterVolumeSpecName: "istio-data") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:07.433400 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.433374 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:42:07.433534 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.433430 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:42:07.433613 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.433593 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-token" (OuterVolumeSpecName: "istio-token") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:42:07.433729 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.433712 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-kube-api-access-shwvz" (OuterVolumeSpecName: "kube-api-access-shwvz") pod "8fbca4bd-0e73-49bf-b821-dfa23699e4a5" (UID: "8fbca4bd-0e73-49bf-b821-dfa23699e4a5"). InnerVolumeSpecName "kube-api-access-shwvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:42:07.532509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.532466 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shwvz\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-kube-api-access-shwvz\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:07.532509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.532500 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:07.532509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.532511 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:07.532509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.532520 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:07.532786 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.532529 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:07.532786 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.532537 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:07.532786 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:07.532544 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fbca4bd-0e73-49bf-b821-dfa23699e4a5-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:42:08.022916 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.022886 2586 generic.go:356] "Generic (PLEG): container finished" podID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerID="b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0" exitCode=0 Jul 09 17:42:08.023109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.022940 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" Jul 09 17:42:08.023109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.022956 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" event={"ID":"8fbca4bd-0e73-49bf-b821-dfa23699e4a5","Type":"ContainerDied","Data":"b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0"} Jul 09 17:42:08.023109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.023001 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf" event={"ID":"8fbca4bd-0e73-49bf-b821-dfa23699e4a5","Type":"ContainerDied","Data":"3841544d888fdd3c658a2eb1664be0b8b8a9858e4326a32801c8513ff96f2711"} Jul 09 17:42:08.023109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.023023 2586 scope.go:117] "RemoveContainer" containerID="b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0" Jul 09 17:42:08.032807 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.032789 2586 scope.go:117] "RemoveContainer" containerID="02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38" Jul 09 17:42:08.040987 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.040968 2586 scope.go:117] "RemoveContainer" containerID="b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0" Jul 09 17:42:08.041266 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:42:08.041244 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0\": container with ID starting with b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0 not found: ID does not exist" containerID="b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0" Jul 09 17:42:08.041316 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.041275 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0"} err="failed to get container status \"b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0\": rpc error: code = NotFound desc = could not find container \"b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0\": container with ID starting with b80d61a7c0db0f2945a3a0b49264409a06b990ccdb3947b229a549c4934e99f0 not found: ID does not exist" Jul 09 17:42:08.041316 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.041293 2586 scope.go:117] "RemoveContainer" containerID="02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38" Jul 09 17:42:08.041705 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:42:08.041684 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38\": container with ID starting with 02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38 not found: ID does not exist" containerID="02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38" Jul 09 17:42:08.041790 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.041711 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38"} err="failed to get container status \"02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38\": rpc error: code = NotFound desc = could not find container \"02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38\": container with ID starting with 02c046e53ed5cd6280a8aef1eb6ac5e1c207dc0dcd3fa974e5f3c163b6be0a38 not found: ID does not exist" Jul 09 17:42:08.072433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.072406 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf"] Jul 09 17:42:08.087423 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.087386 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b6bea1-openshift-default-5ffb9fc7f5-wrskf"] Jul 09 17:42:08.672344 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:08.672307 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" path="/var/lib/kubelet/pods/8fbca4bd-0e73-49bf-b821-dfa23699e4a5/volumes" Jul 09 17:42:11.133800 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.133758 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq"] Jul 09 17:42:11.134202 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.134193 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" Jul 09 17:42:11.134248 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.134204 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" Jul 09 17:42:11.134248 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.134231 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" Jul 09 17:42:11.134248 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.134236 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" Jul 09 17:42:11.134357 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.134302 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" Jul 09 17:42:11.134357 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.134313 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fbca4bd-0e73-49bf-b821-dfa23699e4a5" containerName="istio-proxy" Jul 09 17:42:11.139176 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.139152 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.142294 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.142272 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f62372-openshift-default-dockercfg-dwjj8\"" Jul 09 17:42:11.148359 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.148334 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq"] Jul 09 17:42:11.272384 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272345 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-certs\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.272384 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272384 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-socket\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.272651 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272410 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-istio-token\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.272651 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272477 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75274cff-6041-498e-b575-8840fe65a784-istiod-ca-cert\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.272651 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272501 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnvcf\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-kube-api-access-qnvcf\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.272651 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272523 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75274cff-6041-498e-b575-8840fe65a784-istio-podinfo\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.272788 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272657 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-envoy\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.272788 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272699 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-data\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.272788 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.272771 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-credential-socket\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374081 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374046 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-certs\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374081 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374084 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-socket\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374315 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374107 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-istio-token\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374315 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374219 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75274cff-6041-498e-b575-8840fe65a784-istiod-ca-cert\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374315 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374260 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnvcf\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-kube-api-access-qnvcf\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374315 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374301 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75274cff-6041-498e-b575-8840fe65a784-istio-podinfo\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374594 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374420 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-envoy\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374594 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374435 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-socket\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374594 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374492 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-data\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374594 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374540 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-certs\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374594 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374580 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-credential-socket\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374862 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374721 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-data\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.374926 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.374876 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-credential-socket\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.375045 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.375023 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75274cff-6041-498e-b575-8840fe65a784-istiod-ca-cert\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.376777 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.376753 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-envoy\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.376976 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.376960 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75274cff-6041-498e-b575-8840fe65a784-istio-podinfo\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.382782 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.382760 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-istio-token\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.382970 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.382951 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnvcf\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-kube-api-access-qnvcf\") pod \"e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.452158 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.452071 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:11.600771 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.600741 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq"] Jul 09 17:42:11.602691 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:42:11.602660 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75274cff_6041_498e_b575_8840fe65a784.slice/crio-895a1d39147abecdd078d4ae295af3291d44d150b7f0bf51a5d13c1160984a31 WatchSource:0}: Error finding container 895a1d39147abecdd078d4ae295af3291d44d150b7f0bf51a5d13c1160984a31: Status 404 returned error can't find the container with id 895a1d39147abecdd078d4ae295af3291d44d150b7f0bf51a5d13c1160984a31 Jul 09 17:42:11.605229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.605187 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:11.605320 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.605288 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:11.605365 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:11.605330 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:12.046306 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:12.046270 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" event={"ID":"75274cff-6041-498e-b575-8840fe65a784","Type":"ContainerStarted","Data":"1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b"} Jul 09 17:42:12.046544 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:12.046332 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" event={"ID":"75274cff-6041-498e-b575-8840fe65a784","Type":"ContainerStarted","Data":"895a1d39147abecdd078d4ae295af3291d44d150b7f0bf51a5d13c1160984a31"} Jul 09 17:42:12.067099 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:12.067032 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" podStartSLOduration=1.067015317 podStartE2EDuration="1.067015317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:42:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:42:12.066068987 +0000 UTC m=+1991.995514500" watchObservedRunningTime="2026-07-09 17:42:12.067015317 +0000 UTC m=+1991.996460829" Jul 09 17:42:12.452597 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:12.452508 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:12.454109 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:12.454087 2586 patch_prober.go:28] interesting pod/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 17:42:12.454171 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:12.454136 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 17:42:13.457198 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:13.457166 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:14.057924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:14.057896 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:14.059097 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:14.059079 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:20.353937 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:20.353900 2586 patch_prober.go:28] interesting pod/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:37714->10.134.0.50:15021: read: connection reset by peer" start-of-body= Jul 09 17:42:20.354358 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:20.353972 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:37714->10.134.0.50:15021: read: connection reset by peer" Jul 09 17:42:21.106916 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:21.106875 2586 generic.go:356] "Generic (PLEG): container finished" podID="7b972c82-e791-4e71-961a-8eda14096bf1" containerID="631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7" exitCode=0 Jul 09 17:42:21.107214 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:21.106925 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" event={"ID":"7b972c82-e791-4e71-961a-8eda14096bf1","Type":"ContainerDied","Data":"631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7"} Jul 09 17:42:21.107703 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:21.107681 2586 scope.go:117] "RemoveContainer" containerID="631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7" Jul 09 17:42:21.108893 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:21.108855 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:21.109025 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:21.108950 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:21.109025 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:21.108993 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:22.121543 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:22.121499 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" event={"ID":"7b972c82-e791-4e71-961a-8eda14096bf1","Type":"ContainerStarted","Data":"e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048"} Jul 09 17:42:22.328965 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:22.328927 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:22.329199 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:22.329173 2586 patch_prober.go:28] interesting pod/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 09 17:42:22.329271 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:22.329226 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 09 17:42:23.329359 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:23.329327 2586 patch_prober.go:28] interesting pod/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 09 17:42:23.329781 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:23.329381 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 09 17:42:24.329298 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:24.329258 2586 patch_prober.go:28] interesting pod/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 09 17:42:24.329510 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:24.329315 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 09 17:42:25.329029 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:25.328990 2586 patch_prober.go:28] interesting pod/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 09 17:42:25.329219 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:25.329050 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 09 17:42:26.331666 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:26.331640 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:26.332092 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:26.331994 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:26.340603 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:26.340577 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:42:27.153884 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:27.153777 2586 generic.go:356] "Generic (PLEG): container finished" podID="75274cff-6041-498e-b575-8840fe65a784" containerID="1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b" exitCode=0 Jul 09 17:42:27.153884 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:27.153854 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" event={"ID":"75274cff-6041-498e-b575-8840fe65a784","Type":"ContainerDied","Data":"1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b"} Jul 09 17:42:27.154379 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:27.154363 2586 scope.go:117] "RemoveContainer" containerID="1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b" Jul 09 17:42:27.155320 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:27.155297 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:27.155411 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:27.155359 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:27.155411 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:27.155389 2586 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 17:42:28.161793 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:28.161759 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" event={"ID":"75274cff-6041-498e-b575-8840fe65a784","Type":"ContainerStarted","Data":"3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d"} Jul 09 17:42:28.452582 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:28.452483 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:28.453737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:28.453708 2586 patch_prober.go:28] interesting pod/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 17:42:28.453865 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:28.453773 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 17:42:29.058551 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:29.058495 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:29.452707 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:29.452615 2586 patch_prober.go:28] interesting pod/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 17:42:29.452707 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:29.452673 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 17:42:30.452880 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:30.452842 2586 patch_prober.go:28] interesting pod/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 17:42:30.453294 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:30.452900 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 17:42:31.453347 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:31.453309 2586 patch_prober.go:28] interesting pod/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body= Jul 09 17:42:31.453877 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:31.453370 2586 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" Jul 09 17:42:32.456382 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:32.456350 2586 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:42:32.457505 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:42:32.457486 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:44:00.687479 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:44:00.687424 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:44:00.696942 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:44:00.696921 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:48:24.095509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:24.095473 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v"] Jul 09 17:48:24.096092 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:24.095744 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" containerID="cri-o://e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048" gracePeriod=30 Jul 09 17:48:26.334210 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:26.334166 2586 patch_prober.go:28] interesting pod/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 17:48:26.334636 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:26.334257 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 17:48:29.350147 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.350117 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:48:29.396234 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396152 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-istio-token\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396234 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396191 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b972c82-e791-4e71-961a-8eda14096bf1-istiod-ca-cert\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396234 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396213 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79ckk\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-kube-api-access-79ckk\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396277 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-data\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396509 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396298 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-socket\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396616 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396501 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-credential-socket\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396673 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396619 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:29.396673 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396662 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-envoy\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396778 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396667 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b972c82-e791-4e71-961a-8eda14096bf1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:48:29.396778 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396695 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b972c82-e791-4e71-961a-8eda14096bf1-istio-podinfo\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396778 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396720 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-certs\") pod \"7b972c82-e791-4e71-961a-8eda14096bf1\" (UID: \"7b972c82-e791-4e71-961a-8eda14096bf1\") " Jul 09 17:48:29.396916 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.396809 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:29.397143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.397072 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:29.397143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.397090 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:29.397143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.397103 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:29.397143 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.397121 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b972c82-e791-4e71-961a-8eda14096bf1-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:29.397433 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.397214 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:29.398835 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.398808 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-kube-api-access-79ckk" (OuterVolumeSpecName: "kube-api-access-79ckk") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "kube-api-access-79ckk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:48:29.399008 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.398988 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:48:29.399083 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.399036 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:29.399126 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.399107 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b972c82-e791-4e71-961a-8eda14096bf1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b972c82-e791-4e71-961a-8eda14096bf1" (UID: "7b972c82-e791-4e71-961a-8eda14096bf1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:48:29.497963 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.497926 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:29.497963 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.497957 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b972c82-e791-4e71-961a-8eda14096bf1-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:29.497963 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.497965 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:29.497963 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.497974 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:29.498237 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.497982 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79ckk\" (UniqueName: \"kubernetes.io/projected/7b972c82-e791-4e71-961a-8eda14096bf1-kube-api-access-79ckk\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:29.498237 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:29.497991 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b972c82-e791-4e71-961a-8eda14096bf1-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:30.082059 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.082022 2586 generic.go:356] "Generic (PLEG): container finished" podID="7b972c82-e791-4e71-961a-8eda14096bf1" containerID="e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048" exitCode=0 Jul 09 17:48:30.082275 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.082080 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" Jul 09 17:48:30.082275 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.082083 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" event={"ID":"7b972c82-e791-4e71-961a-8eda14096bf1","Type":"ContainerDied","Data":"e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048"} Jul 09 17:48:30.082275 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.082200 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v" event={"ID":"7b972c82-e791-4e71-961a-8eda14096bf1","Type":"ContainerDied","Data":"8bdc485a48f482649841e8d6611a7bdaf184df9479654c7b288b965bbda0060d"} Jul 09 17:48:30.082275 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.082222 2586 scope.go:117] "RemoveContainer" containerID="e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048" Jul 09 17:48:30.091805 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.091784 2586 scope.go:117] "RemoveContainer" containerID="631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7" Jul 09 17:48:30.102386 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.102364 2586 scope.go:117] "RemoveContainer" containerID="e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048" Jul 09 17:48:30.102693 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:48:30.102668 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048\": container with ID starting with e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048 not found: ID does not exist" containerID="e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048" Jul 09 17:48:30.102818 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.102700 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048"} err="failed to get container status \"e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048\": rpc error: code = NotFound desc = could not find container \"e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048\": container with ID starting with e64700f3bf27095eac2688603994ac1ca0ec386951b243ba3467543f4e0dd048 not found: ID does not exist" Jul 09 17:48:30.102818 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.102720 2586 scope.go:117] "RemoveContainer" containerID="631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7" Jul 09 17:48:30.103078 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:48:30.103051 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7\": container with ID starting with 631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7 not found: ID does not exist" containerID="631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7" Jul 09 17:48:30.103194 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.103083 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7"} err="failed to get container status \"631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7\": rpc error: code = NotFound desc = could not find container \"631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7\": container with ID starting with 631d5dfc55d40f591d8a1054b680d9a565072104cb25ccbcf6baea49a5f3d0f7 not found: ID does not exist" Jul 09 17:48:30.106079 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.106057 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v"] Jul 09 17:48:30.113200 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.113176 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-77a679-openshift-default-77594f5467-7fl9v"] Jul 09 17:48:30.672136 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:30.672102 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" path="/var/lib/kubelet/pods/7b972c82-e791-4e71-961a-8eda14096bf1/volumes" Jul 09 17:48:31.160399 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:31.160361 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq"] Jul 09 17:48:31.160668 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:31.160640 2586 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" containerID="cri-o://3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d" gracePeriod=30 Jul 09 17:48:32.459218 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:32.459176 2586 patch_prober.go:28] interesting pod/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 17:48:32.459708 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:32.459254 2586 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 17:48:36.418662 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.418632 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:48:36.469640 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469599 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-istio-token\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.469825 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469648 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-data\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.469825 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469678 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-envoy\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.469825 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469729 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75274cff-6041-498e-b575-8840fe65a784-istio-podinfo\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.469825 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469765 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnvcf\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-kube-api-access-qnvcf\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.469825 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469825 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-certs\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.470099 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469862 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-credential-socket\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.470099 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469909 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-socket\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.470099 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.469943 2586 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75274cff-6041-498e-b575-8840fe65a784-istiod-ca-cert\") pod \"75274cff-6041-498e-b575-8840fe65a784\" (UID: \"75274cff-6041-498e-b575-8840fe65a784\") " Jul 09 17:48:36.470298 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.470270 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:36.470394 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.470368 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:36.470559 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.470520 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:36.470624 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.470583 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-data" (OuterVolumeSpecName: "istio-data") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:36.470695 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.470662 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75274cff-6041-498e-b575-8840fe65a784-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:48:36.472016 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.471994 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-istio-token" (OuterVolumeSpecName: "istio-token") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:48:36.472134 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.472045 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75274cff-6041-498e-b575-8840fe65a784-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 17:48:36.472261 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.472244 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:36.472358 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.472341 2586 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-kube-api-access-qnvcf" (OuterVolumeSpecName: "kube-api-access-qnvcf") pod "75274cff-6041-498e-b575-8840fe65a784" (UID: "75274cff-6041-498e-b575-8840fe65a784"). InnerVolumeSpecName "kube-api-access-qnvcf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:48:36.570795 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570758 2586 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-istio-token\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:36.570795 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570788 2586 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-data\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:36.570795 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570799 2586 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-istio-envoy\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:36.571052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570810 2586 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75274cff-6041-498e-b575-8840fe65a784-istio-podinfo\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:36.571052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570824 2586 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnvcf\" (UniqueName: \"kubernetes.io/projected/75274cff-6041-498e-b575-8840fe65a784-kube-api-access-qnvcf\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:36.571052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570837 2586 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-certs\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:36.571052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570849 2586 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-credential-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:36.571052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570860 2586 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75274cff-6041-498e-b575-8840fe65a784-workload-socket\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:36.571052 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:36.570872 2586 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75274cff-6041-498e-b575-8840fe65a784-istiod-ca-cert\") on node \"ip-10-0-135-186.ec2.internal\" DevicePath \"\"" Jul 09 17:48:37.120287 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.120249 2586 generic.go:356] "Generic (PLEG): container finished" podID="75274cff-6041-498e-b575-8840fe65a784" containerID="3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d" exitCode=0 Jul 09 17:48:37.120496 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.120293 2586 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" Jul 09 17:48:37.120496 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.120328 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" event={"ID":"75274cff-6041-498e-b575-8840fe65a784","Type":"ContainerDied","Data":"3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d"} Jul 09 17:48:37.120496 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.120391 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq" event={"ID":"75274cff-6041-498e-b575-8840fe65a784","Type":"ContainerDied","Data":"895a1d39147abecdd078d4ae295af3291d44d150b7f0bf51a5d13c1160984a31"} Jul 09 17:48:37.120496 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.120420 2586 scope.go:117] "RemoveContainer" containerID="3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d" Jul 09 17:48:37.129402 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.129380 2586 scope.go:117] "RemoveContainer" containerID="1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b" Jul 09 17:48:37.138522 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.138497 2586 scope.go:117] "RemoveContainer" containerID="3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d" Jul 09 17:48:37.138786 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:48:37.138768 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d\": container with ID starting with 3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d not found: ID does not exist" containerID="3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d" Jul 09 17:48:37.138858 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.138794 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d"} err="failed to get container status \"3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d\": rpc error: code = NotFound desc = could not find container \"3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d\": container with ID starting with 3d8cb9489eef40fadacc7756ad30af234cab1895bc8a8886c67b827e97163f1d not found: ID does not exist" Jul 09 17:48:37.138858 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.138814 2586 scope.go:117] "RemoveContainer" containerID="1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b" Jul 09 17:48:37.139028 ip-10-0-135-186 kubenswrapper[2586]: E0709 17:48:37.139008 2586 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b\": container with ID starting with 1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b not found: ID does not exist" containerID="1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b" Jul 09 17:48:37.139082 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.139032 2586 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b"} err="failed to get container status \"1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b\": rpc error: code = NotFound desc = could not find container \"1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b\": container with ID starting with 1e79acb69298462a0496520f8a3cbe198e19664457f7622ee5cac2deed6e2e4b not found: ID does not exist" Jul 09 17:48:37.140398 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.140378 2586 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq"] Jul 09 17:48:37.144789 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.144769 2586 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f62372-openshift-default-864fd7757f-rnjgq"] Jul 09 17:48:37.339843 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.339815 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849f76587-4p6ln_1d9dbdf0-9a28-4a2f-b110-2e767aeb57db/copy-manifests/0.log" Jul 09 17:48:37.347756 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.347733 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849f76587-4p6ln_1d9dbdf0-9a28-4a2f-b110-2e767aeb57db/manager/0.log" Jul 09 17:48:37.958006 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:37.957976 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-589b48b676-j4xhp_d3427597-e9e5-462b-8ea0-d0fc18597d5a/maas-api/0.log" Jul 09 17:48:38.672144 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:38.672107 2586 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75274cff-6041-498e-b575-8840fe65a784" path="/var/lib/kubelet/pods/75274cff-6041-498e-b575-8840fe65a784/volumes" Jul 09 17:48:40.734090 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:40.734055 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zdnfj_89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0/istio-proxy/0.log" Jul 09 17:48:40.857737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:40.857701 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fss2k_c1939905-ac25-4557-b8f8-5a297af75629/discovery/0.log" Jul 09 17:48:41.217860 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:41.217827 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zpstw_17015692-b9b8-4cd2-b976-100fa0044291/istio-proxy/1.log" Jul 09 17:48:41.465432 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:41.465401 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zv5sg_4fbc7353-a95a-4d78-a47e-8f387cebb21c/payload-processing/0.log" Jul 09 17:48:42.056502 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:42.056425 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-kf855_7b544854-8484-4e55-af57-b7d0bc5f566b/main/0.log" Jul 09 17:48:42.288268 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:42.288235 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxpgps_859fc1ab-c51c-4f96-ae1f-1ee8633de13f/main/0.log" Jul 09 17:48:49.401176 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:49.401141 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-txnlq_c1a4c79c-6a26-4c67-8ae9-7ecad2fab6c4/global-pull-secret-syncer/0.log" Jul 09 17:48:49.486529 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:49.486491 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l448f_9204fadf-bf81-4ebb-bf8f-75b94fa2db53/konnectivity-agent/0.log" Jul 09 17:48:49.594496 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:49.594465 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-186.ec2.internal_b45ab77f7adac80ae4f46c80d27c83b4/haproxy/0.log" Jul 09 17:48:56.213414 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.213380 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bd62717-556d-42cf-9f78-7269904a5684/alertmanager/0.log" Jul 09 17:48:56.239494 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.239465 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bd62717-556d-42cf-9f78-7269904a5684/config-reloader/0.log" Jul 09 17:48:56.264391 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.264363 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bd62717-556d-42cf-9f78-7269904a5684/kube-rbac-proxy-web/0.log" Jul 09 17:48:56.287187 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.287157 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bd62717-556d-42cf-9f78-7269904a5684/kube-rbac-proxy/0.log" Jul 09 17:48:56.308017 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.307987 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bd62717-556d-42cf-9f78-7269904a5684/kube-rbac-proxy-metric/0.log" Jul 09 17:48:56.330542 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.330496 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bd62717-556d-42cf-9f78-7269904a5684/prom-label-proxy/0.log" Jul 09 17:48:56.354082 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.354042 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4bd62717-556d-42cf-9f78-7269904a5684/init-config-reloader/0.log" Jul 09 17:48:56.425461 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.425388 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-hmsbr_1b401422-cf54-4944-96d1-2853d2f9b8f0/kube-state-metrics/0.log" Jul 09 17:48:56.441231 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.441197 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-hmsbr_1b401422-cf54-4944-96d1-2853d2f9b8f0/kube-rbac-proxy-main/0.log" Jul 09 17:48:56.462822 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.462789 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-hmsbr_1b401422-cf54-4944-96d1-2853d2f9b8f0/kube-rbac-proxy-self/0.log" Jul 09 17:48:56.488114 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.488031 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-77597554fc-57cgb_9f41f1d0-2a3c-446f-aaff-17490f9017a8/metrics-server/0.log" Jul 09 17:48:56.511460 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.511400 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-dtzns_422fa30b-b3e1-42e8-9c32-4a34b97f53cd/monitoring-plugin/0.log" Jul 09 17:48:56.623656 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.623624 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ngcwp_02bfa02d-1348-4b6c-981e-798b801ba3b1/node-exporter/0.log" Jul 09 17:48:56.643234 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.643207 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ngcwp_02bfa02d-1348-4b6c-981e-798b801ba3b1/kube-rbac-proxy/0.log" Jul 09 17:48:56.662232 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.662209 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ngcwp_02bfa02d-1348-4b6c-981e-798b801ba3b1/init-textfile/0.log" Jul 09 17:48:56.788042 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.787942 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-nhbbd_7257f390-a323-4bb0-889e-709297bcb63f/kube-rbac-proxy-main/0.log" Jul 09 17:48:56.811601 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.811573 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-nhbbd_7257f390-a323-4bb0-889e-709297bcb63f/kube-rbac-proxy-self/0.log" Jul 09 17:48:56.839381 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.839347 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-nhbbd_7257f390-a323-4bb0-889e-709297bcb63f/openshift-state-metrics/0.log" Jul 09 17:48:56.898067 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.898037 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1f109601-c3bb-431e-9dfa-97a15be272c7/prometheus/0.log" Jul 09 17:48:56.923089 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.923047 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1f109601-c3bb-431e-9dfa-97a15be272c7/config-reloader/0.log" Jul 09 17:48:56.951330 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.951304 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1f109601-c3bb-431e-9dfa-97a15be272c7/thanos-sidecar/0.log" Jul 09 17:48:56.976158 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:56.976124 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1f109601-c3bb-431e-9dfa-97a15be272c7/kube-rbac-proxy-web/0.log" Jul 09 17:48:57.006883 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.006855 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1f109601-c3bb-431e-9dfa-97a15be272c7/kube-rbac-proxy/0.log" Jul 09 17:48:57.029157 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.029131 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1f109601-c3bb-431e-9dfa-97a15be272c7/kube-rbac-proxy-thanos/0.log" Jul 09 17:48:57.051594 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.051506 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1f109601-c3bb-431e-9dfa-97a15be272c7/init-config-reloader/0.log" Jul 09 17:48:57.082818 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.082789 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rj9v6_eb64569e-f3cb-4b00-bba8-52ba45a65c90/prometheus-operator/0.log" Jul 09 17:48:57.107193 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.107161 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rj9v6_eb64569e-f3cb-4b00-bba8-52ba45a65c90/kube-rbac-proxy/0.log" Jul 09 17:48:57.136938 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.136907 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-f75lc_6d7ad4a5-130a-469d-a834-49fa4ef6192b/prometheus-operator-admission-webhook/0.log" Jul 09 17:48:57.180253 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.180216 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f79d4fbc9-pmfsw_fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8/telemeter-client/0.log" Jul 09 17:48:57.203591 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.203566 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f79d4fbc9-pmfsw_fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8/reload/0.log" Jul 09 17:48:57.227225 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.227198 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f79d4fbc9-pmfsw_fd8e3c76-9198-482e-92ab-f3fb8ea0bfa8/kube-rbac-proxy/0.log" Jul 09 17:48:57.294546 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.294516 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c8d57cbb5-s7vtg_83b49f60-733f-4618-893a-e2b2f61969f1/thanos-query/0.log" Jul 09 17:48:57.317745 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.317664 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c8d57cbb5-s7vtg_83b49f60-733f-4618-893a-e2b2f61969f1/kube-rbac-proxy-web/0.log" Jul 09 17:48:57.342165 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.342138 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c8d57cbb5-s7vtg_83b49f60-733f-4618-893a-e2b2f61969f1/kube-rbac-proxy/0.log" Jul 09 17:48:57.359953 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.359927 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c8d57cbb5-s7vtg_83b49f60-733f-4618-893a-e2b2f61969f1/prom-label-proxy/0.log" Jul 09 17:48:57.392649 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.392610 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c8d57cbb5-s7vtg_83b49f60-733f-4618-893a-e2b2f61969f1/kube-rbac-proxy-rules/0.log" Jul 09 17:48:57.415156 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:57.415131 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c8d57cbb5-s7vtg_83b49f60-733f-4618-893a-e2b2f61969f1/kube-rbac-proxy-metrics/0.log" Jul 09 17:48:59.851235 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:48:59.851192 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-b9t6l_8e47f7c4-b0c9-4597-95c9-678ba1b3866d/download-server/0.log" Jul 09 17:49:00.182001 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.181917 2586 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d"] Jul 09 17:49:00.182483 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182464 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" Jul 09 17:49:00.182557 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182486 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" Jul 09 17:49:00.182557 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182509 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" Jul 09 17:49:00.182557 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182516 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" Jul 09 17:49:00.182557 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182525 2586 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" Jul 09 17:49:00.182557 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182531 2586 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" Jul 09 17:49:00.182728 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182614 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" Jul 09 17:49:00.182728 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182626 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="75274cff-6041-498e-b575-8840fe65a784" containerName="istio-proxy" Jul 09 17:49:00.182728 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182631 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" Jul 09 17:49:00.182728 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.182637 2586 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b972c82-e791-4e71-961a-8eda14096bf1" containerName="istio-proxy" Jul 09 17:49:00.187536 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.187517 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.190755 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.190731 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gdggh\"/\"kube-root-ca.crt\"" Jul 09 17:49:00.190891 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.190770 2586 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gdggh\"/\"openshift-service-ca.crt\"" Jul 09 17:49:00.192262 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.192242 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gdggh\"/\"default-dockercfg-fngdp\"" Jul 09 17:49:00.196169 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.196144 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d"] Jul 09 17:49:00.208407 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.208309 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s58np\" (UniqueName: \"kubernetes.io/projected/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-kube-api-access-s58np\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.208407 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.208388 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-podres\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.208720 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.208467 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-lib-modules\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.208720 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.208509 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-sys\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.208720 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.208535 2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-proc\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309480 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309422 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s58np\" (UniqueName: \"kubernetes.io/projected/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-kube-api-access-s58np\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309509 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-podres\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309550 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-lib-modules\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309575 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-sys\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309598 2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-proc\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309673 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-sys\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309683 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-lib-modules\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309692 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309693 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-proc\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.309982 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.309685 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-podres\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.317737 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.317706 2586 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s58np\" (UniqueName: \"kubernetes.io/projected/ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9-kube-api-access-s58np\") pod \"perf-node-gather-daemonset-b6t4d\" (UID: \"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.501964 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.501936 2586 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gdggh\"/\"default-dockercfg-fngdp\"" Jul 09 17:49:00.509273 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.509249 2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:00.648571 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.645368 2586 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d"] Jul 09 17:49:00.653287 ip-10-0-135-186 kubenswrapper[2586]: W0709 17:49:00.653255 2586 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podccd610bf_fe82_4b1e_9cdc_a7b83c0a7dd9.slice/crio-d1a1fe87f30197d8863f68be80e3f8d4f36971c55075db2ef45ea776f69f2ef5 WatchSource:0}: Error finding container d1a1fe87f30197d8863f68be80e3f8d4f36971c55075db2ef45ea776f69f2ef5: Status 404 returned error can't find the container with id d1a1fe87f30197d8863f68be80e3f8d4f36971c55075db2ef45ea776f69f2ef5 Jul 09 17:49:00.720868 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.720840 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:49:00.730094 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:00.730068 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcs4h_7cdfdc66-e339-48f1-bdb0-d930d999d0fc/ovn-acl-logging/0.log" Jul 09 17:49:01.243371 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:01.243334 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" event={"ID":"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9","Type":"ContainerStarted","Data":"ceae26180b7b0d260310fd677970aaf78e109382b0bdf2f4f4ddb353df54bf02"} Jul 09 17:49:01.243878 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:01.243395 2586 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" event={"ID":"ccd610bf-fe82-4b1e-9cdc-a7b83c0a7dd9","Type":"ContainerStarted","Data":"d1a1fe87f30197d8863f68be80e3f8d4f36971c55075db2ef45ea776f69f2ef5"} Jul 09 17:49:01.243878 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:01.243538 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:01.261242 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:01.261190 2586 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" podStartSLOduration=1.261174209 podStartE2EDuration="1.261174209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:49:01.26025433 +0000 UTC m=+2401.189699861" watchObservedRunningTime="2026-07-09 17:49:01.261174209 +0000 UTC m=+2401.190619721" Jul 09 17:49:01.613924 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:01.613805 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jjlnk_0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e/dns/0.log" Jul 09 17:49:01.635989 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:01.635928 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jjlnk_0f4bab7d-8135-4bd5-a2ad-4e62f5fc727e/kube-rbac-proxy/0.log" Jul 09 17:49:01.711331 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:01.711298 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kshcf_dbf9d27e-b21d-4436-8502-3543544a2fe1/dns-node-resolver/0.log" Jul 09 17:49:02.293027 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:02.292994 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5x9qp_b238ba72-80c9-42b3-a046-cc4cc546e62d/node-ca/0.log" Jul 09 17:49:03.274253 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:03.274219 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zdnfj_89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0/istio-proxy/0.log" Jul 09 17:49:03.470449 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:03.470402 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fss2k_c1939905-ac25-4557-b8f8-5a297af75629/discovery/0.log" Jul 09 17:49:03.548550 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:03.548431 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zpstw_17015692-b9b8-4cd2-b976-100fa0044291/istio-proxy/0.log" Jul 09 17:49:03.607249 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:03.607212 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zpstw_17015692-b9b8-4cd2-b976-100fa0044291/istio-proxy/1.log" Jul 09 17:49:03.678397 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:03.678362 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zv5sg_4fbc7353-a95a-4d78-a47e-8f387cebb21c/payload-processing/0.log" Jul 09 17:49:04.422976 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:04.422946 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-f9xp9_bc2f9b71-3b62-4813-af2b-01a5efde8459/serve-healthcheck-canary/0.log" Jul 09 17:49:04.980792 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:04.980759 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2h2b_da168133-4e14-41ca-a48b-590f0a5fc054/kube-rbac-proxy/0.log" Jul 09 17:49:05.002190 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:05.002163 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2h2b_da168133-4e14-41ca-a48b-590f0a5fc054/exporter/0.log" Jul 09 17:49:05.027135 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:05.027104 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2h2b_da168133-4e14-41ca-a48b-590f0a5fc054/extractor/0.log" Jul 09 17:49:07.259080 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:07.259052 2586 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-b6t4d" Jul 09 17:49:07.491086 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:07.491057 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849f76587-4p6ln_1d9dbdf0-9a28-4a2f-b110-2e767aeb57db/manager/0.log" Jul 09 17:49:07.511582 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:07.511495 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849f76587-4p6ln_1d9dbdf0-9a28-4a2f-b110-2e767aeb57db/copy-manifests/0.log" Jul 09 17:49:09.009370 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:09.009286 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59885db7cf-mxhq9_ddc21b4e-fdc2-43cf-82ab-386404103633/manager/0.log" Jul 09 17:49:09.036299 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:09.036272 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jgxgr_b7cc259f-0af4-4177-b4fc-328d68e08936/openshift-lws-operator/0.log" Jul 09 17:49:09.998376 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:09.998346 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zdnfj_89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0/istio-proxy/0.log" Jul 09 17:49:10.196229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:10.196135 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fss2k_c1939905-ac25-4557-b8f8-5a297af75629/discovery/0.log" Jul 09 17:49:10.246483 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:10.246429 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zpstw_17015692-b9b8-4cd2-b976-100fa0044291/istio-proxy/0.log" Jul 09 17:49:10.308739 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:10.308710 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zpstw_17015692-b9b8-4cd2-b976-100fa0044291/istio-proxy/1.log" Jul 09 17:49:10.371608 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:10.371574 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zv5sg_4fbc7353-a95a-4d78-a47e-8f387cebb21c/payload-processing/0.log" Jul 09 17:49:11.636386 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:11.636348 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678zdnfj_89ef2fdd-8b6e-4f32-8de2-61b9ee13b6d0/istio-proxy/0.log" Jul 09 17:49:11.832185 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:11.832109 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fss2k_c1939905-ac25-4557-b8f8-5a297af75629/discovery/0.log" Jul 09 17:49:11.884167 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:11.884139 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zpstw_17015692-b9b8-4cd2-b976-100fa0044291/istio-proxy/0.log" Jul 09 17:49:11.946229 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:11.946199 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-zpstw_17015692-b9b8-4cd2-b976-100fa0044291/istio-proxy/1.log" Jul 09 17:49:12.003630 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:12.003598 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-zv5sg_4fbc7353-a95a-4d78-a47e-8f387cebb21c/payload-processing/0.log" Jul 09 17:49:15.707376 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:15.707345 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qwprr_13e9a0cf-c2b6-4824-8447-ec09c276fd53/kube-multus-additional-cni-plugins/0.log" Jul 09 17:49:15.729397 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:15.729368 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qwprr_13e9a0cf-c2b6-4824-8447-ec09c276fd53/egress-router-binary-copy/0.log" Jul 09 17:49:15.749697 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:15.749662 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qwprr_13e9a0cf-c2b6-4824-8447-ec09c276fd53/cni-plugins/0.log" Jul 09 17:49:15.770478 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:15.770432 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qwprr_13e9a0cf-c2b6-4824-8447-ec09c276fd53/bond-cni-plugin/0.log" Jul 09 17:49:15.791304 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:15.791278 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qwprr_13e9a0cf-c2b6-4824-8447-ec09c276fd53/routeoverride-cni/0.log" Jul 09 17:49:15.812453 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:15.812395 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qwprr_13e9a0cf-c2b6-4824-8447-ec09c276fd53/whereabouts-cni-bincopy/0.log" Jul 09 17:49:15.834057 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:15.834028 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qwprr_13e9a0cf-c2b6-4824-8447-ec09c276fd53/whereabouts-cni/0.log" Jul 09 17:49:16.068033 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:16.068002 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tklf4_0c29139d-04e4-45ee-a125-a01247b1f99e/kube-multus/0.log" Jul 09 17:49:16.089599 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:16.089568 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hbwvp_ea9dd5df-3bc5-4eb7-a428-a93987879809/network-metrics-daemon/0.log" Jul 09 17:49:16.107091 ip-10-0-135-186 kubenswrapper[2586]: I0709 17:49:16.107057 2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hbwvp_ea9dd5df-3bc5-4eb7-a428-a93987879809/kube-rbac-proxy/0.log"